literature review on learning management system

• Previous Article
• Next Article

A review of learning management systems (LMS) framework towards the element of outcome based education (OBE)

[email protected]

• Article contents
• Figures & tables
• Supplementary Data
• Peer Review
• Reprints and Permissions
• Cite Icon Cite
• Search Site

Haziman Madiah , Rosmayati Mohemad; A review of learning management systems (LMS) framework towards the element of outcome based education (OBE). AIP Conf. Proc. 23 March 2023; 2484 (1): 060014. https://doi.org/10.1063/5.0113769

Download citation file:

• Ris (Zotero)
• Reference Manager

The emergence of the Learning Management System (LMS) has significantly enhanced the teaching and learning process by centralising the management, organisation, delivery, and reporting of educational content and student outcomes. Moving towards outcome-based education (OBE), a LMS that is capable of supporting the delivery of OBE courses can facilitate more effective engagement with students, thereby improving students' learning outcomes and accelerating the process of continuous quality improvement. However, there is no well-rounded LMS framework for supporting the implementation of the OBE learning process to the best of our knowledge. Therefore, the objective of this paper is to investigate how well the OBE elements are integrated into the LMS framework and what research gaps exist at the delivery and assessment stages of the OBE course in the context of an LMS. The methodology entails a review of related works on LMS in terms of user management and administration, pedagogy style, usability and learning outcomes, support for learner behaviours and actions, design specification, clear and user-friendly graphical interface, course schedule creation, course administration capability, course content creation tools, rich content types, well-designed course repository, a student's profile, assessment, score, and transcript, learning and instructional design, the ability of users to interact, collaborate, evaluate, and provide feedback, reporting, technical specification and support, and finally, security. A selection of articles is analysed to identify areas for further investigation as well as the limitations of the current LMS. In summary, the findings indicate that none of the currently available LMSs effectively support and capture OBE knowledge during the delivery and assessment stages. As a result, the outcomes are unfavourable and conform to OBE requirements. Additionally, one of the significant challenges that must be addressed is designing and developing the ideal OBE-based LMS to close the gap.

Sign in via your Institution

Citing articles via, publish with us - request a quote.

Sign up for alerts

• Online ISSN 1551-7616
• Print ISSN 0094-243X
• For Researchers
• For Librarians
• For Advertisers
• Our Publishing Partners  
• Physics Today
• Conference Proceedings
• Special Topics

pubs.aip.org

• Privacy Policy
• Terms of Use

Connect with AIP Publishing

This feature is available to subscribers only.

Sign In or Create an Account

• Open access
• Published: 14 October 2020

An analysis of users’ preferences on learning management systems: a case on German versus Spanish students

• Hasan Tinmaz 1 &
• Jin Hwa Lee   ORCID: orcid.org/0000-0001-6205-0634 2  

Smart Learning Environments volume  7 , Article number:  30 ( 2020 ) Cite this article

13k Accesses

10 Citations

1 Altmetric

Metrics details

The recent advancements in information and communication technologies have altered instructional contexts and re-shaped them into smart learning environments. One of the most common practices of these environments are learning management systems (LMS) where the learners and instructors utilize a software platform to fulfill, support and manage instructional activities around predefined objectives. Successful implementations of LMS have brought a variety on its usage from different cultures, genders, age groups or schooling levels. Hence, this study focuses on understanding the role of culture on LMS design, in along with the effects of gender, age and school year variables. The study participants were German ( n  = 83) and Spanish (n = 83) university students attending a fully online course offered by a South Korean university. At the end of the course, the students were asked to fulfill a survey on effective LMS design by pointing which features of LMS were more important for them. The survey included twenty questions on four major design factors; content management (six items), ease of use (five items), communication within LMS (four item) and screen design (five items). The dataset was analyzed by non-parametric statistical techniques around four variables on four dimensions (and their related survey questions). The most important result was insufficiency of one unique LMS design for all students which demonstrates the necessity of student demographics tailored smart systems. Additionally, age and gender variables were not making significant differences on LMS design as much as culture and school year variables. The study also revealed that while German students would appreciate goal-oriented individual learning, Spanish students would value process-oriented group learning with active communication. Furthermore, many features of LMS were highly valued by the freshman students more than other levels. The paper discusses these variables with possible explanations from the literature and depicts implementations for future design practices.

Introduction

Teaching and learning processes have continuously evolved with technological advances. Correspondingly, the rapid development of information and communication technologies has shaped traditional classrooms into smart learning environments. For instance, sharing learning materials online has enabled the learners to study whenever and wherever they want. Online attendance marking systems have also dramatically reduced the amount of time spent by the instructors to check their students’ attendance and thus increased the actual teaching and learning time. Moreover, the assignments or examinations can also be delivered online with appropriate instructions and feedback to support learning outside of scheduled classes. Likewise, there are many other ubiquitous combinations of pedagogical practices supported or facilitated with recent technologies.

Due to increasing number of available smart learning features, it has become indispensable to manage these features for effective and organized instructional processes. Currently, it is commonly seen that educational institutes operate their own Learning Management Systems (LMS) and provide various online smart learning features for a diverse group of students. An LMS is known as a web-based system that possesses an extensive range of pedagogical and course administration tools (Yakubu, 2019 ). Through these educational tools, LMS can facilitate group chats, discussions, document sharing, assignment submission, quizzes, grading and course evaluations (Bove, & A.,, & Conklin, S., 2020 ). Moreover, LMS has a potential to serve students with diverse backgrounds including culture, age or gender.

Previous studies have focused on identifying various learning features of LMS that can influence students’ learning outcomes. However, it seems that the results of previous studies were controversial with inconsistent learning outcomes of the students. One possible reason can be due to the lack of thorough understanding on students’ learning preferences, needs and diverse backgrounds. As the essence of an LMS is to facilitate self-regulated learning (Douglas & Alemanne, 2007 ), there is a need for analyzing and understanding users’ preferences when applying LMS in educational contexts which will serve various learning needs of the students.

As such, this study aims to analyze key factors that can influence users’ preferences on LMS use and gain a deeper understanding of how to maximize the learning outcomes through LMS by considering four essential independent variables; culture, gender, age and school years. The results of this study will contribute to a successful implementation of smart learning in class.

Literature review

The existing learning management systems literature has identified four major factors that need to be considered for successful implementation of LMS in order to fulfill students’ learning needs and expectations. In this part, four major factors which were yielded from the conclusions of the existing literature review were elaborated. These four factors which are relevant with LMS user experiences are culture, gender, age, and school year.

Cultural factor

When the researchers conducted literature review and analysis, it was unfolded that previous studies highlighted the importance of cultural factor in online learning. According to Hunt and Tickner ( 2015 ), culture is defined as “…a complex and multi-dimensional construct that represents the shared values, beliefs, and basic assumptions of groups of people. It includes elements such as language, customs, social behavior, and religion, and it influences how individuals relate to the world…” (p. 27). One of the most widely used models for understanding characteristics of cultural behaviors was developed by Hofstede ( 1986 ). In this model, he highlighted four dimensions of cultural behaviors or characteristics; ‘small versus large power distance’, ‘individualism versus collectivism’, ‘masculinity versus femininity’, and ‘tolerance of uncertainty and ambiguity versus uncertainty avoidance’.

First of all, power distance is defined as “the extent to which the less powerful members of institutions and organizations within a country expect and accept that power is distributed unequally” (p. 61) (Hofstede, Hofstede, & Minkov, 2010 ). For example, learning characteristics in high power distance cultures are more oriented towards one-way, directive, and instructor-based learning (Swierczek & Bechter, 2010 ). The second dimension, individualism versus collectivism, as mentioned by Mercado, Parboteeah, and Zhao ( 2004 ), means “the tendency of members of a society to act as individuals or members of groups, and to which a culture values individual versus collective achievement or well-being” (p. 185). It can be seen that individualistic culture is more results oriented whereas collective culture is more consensus and discussion oriented (Swierczek & Bechter, 2010 ).

In terms of the third dimension, masculinity versus femininity, a masculine culture shows clearly distinct emotional gender roles for men and women whereas feminine culture has overlapped emotional gender roles (Hofstede et al., 2010 ). Hence it seems that the learning characteristics in masculine society are achievement and competition focused rather than being affiliation oriented (Swierczek & Bechter, 2010 ). Lastly, uncertainty avoidance is defined as “the extent to which the members of a culture feel threatened by ambiguous or unknown situations” (p. 191) (Hofstede et al., 2010 ). With high uncertainty avoidance, guided and structured learning is preferred over independent and open-ended learning (Swierczek & Bechter, 2010 ).

Since our study involves German and Spanish students, their cultural differences were reviewed by using the four-dimensional model of Hofstede (Hofstede et al., 2010 ). It was noticed that Germany is a less power distant country than Spain and their relative power distance scores were 35 and 57 respectively. Thus, it is expected that Spanish students are more familiar with a hierarchical learning environment than German students. In terms of individualism versus collectivism, Germany and Spain obtained individualism scores of 67 and 51 respectively showing their different cultural views on individual versus group. It is expected that German students are more oriented towards individual learning and achievement. On the other hand, Spain’s score was the second lowest among the European countries reflecting higher cultural tendency towards collectivism. When masculinity scores are compared, Germany appears to be a masculine society while Spain is a feminine society with relative scores of 66 and 42 respectively. It implies that German students value high performance and competition-based learning whereas Spanish students prefer harmony and non-competitive learning. Finally, uncertainty avoidance scores for both countries seem generally higher than the average. The relative scores for Germany and Spain were 65 and 86 respectively. As Spain scored higher than Germany, it is predicted that Spanish students will be more reluctant to experience changes, ambiguities, and undefined situations in learning.

In addition to identifying learners’ characteristics, the four-dimensional model of Hofstede can be also used to analyze LMS acceptance levels in educational institutes. For example, Asunka ( 2016 ) identified cultural factors responsible for low LMS acceptance levels of university faculty members. Through a participatory action research approach, the study engaged 10 faculty members for one semester. Among the four cultural dimensions of Hofstede’s model, ‘power distance’ was identified as the most influencing factor followed by ‘individualism versus collectivism’ and ‘uncertainty avoidance’. Although this study applied the Hofstede’s model on instructors rather than students, it highlighted significance of cultural factors in LMS implementation. In addition, Tarhini, Hone, Liu, and Tarhini ( 2017 ) revealed that these four cultural dimensions play an important role in students’ technology acceptance level by influencing subjective norms. The study collected data from 569 undergraduate and postgraduate students in Lebanon using e-learning tools.

Although the Hofstede’s model was not directly applied, previous studies have also reported the impact of cultural factors on students’ learning performance and behaviors. For example, Liu, Liu, Lee, and Magjuka ( 2010 ) investigated the impact of cultural differences on international students’ learning performance. This study involved international students from India, China, and Russia who attended an online Master of Business Administration (MBA) program. Through this study, cultural factors including language, communication tool use, plagiarism, time zone differences, and a lack of multicultural contents were suggested as potential barriers for online learning that can affect students’ learning performance. Similarly, cultural influences on learning behaviors were reported by Swierczek and Bechter ( 2010 ). Their study performed qualitative and quantitative analyses of e-learning behaviors of participants from South Asia, East Asia, and Europe. The study results indicated that European students tend to be individualistic and prefer learning by induction whereas South and East Asian students value affiliation and avoid high uncertainty in learning. In addition, East Asian students appeared to be more active and involved in e-learning. The study suggested several cultural factors responsible for different learning behaviors which included language, technology, the role of instructor, and the level of interaction required. In other words, LMS designing and implementation should cater for various learning needs of students that can arise due to different cultural backgrounds.

The existence of cultural factors often creates cultural barriers to limit the potentials of online learning facilitated by LMS. Thus, it is inevitable to identify possible solutions to overcome these barriers. As mentioned above, Asunka ( 2016 ) reported existence of cultural barriers among the instructors that can trigger their anxiety, uncertainty, and indifference towards LMS usage. However, identification of responsible cultural dimensions, regular discussions, and monitoring the outcomes of LMS implementation throughout the semester made positive changes in the instructors’ views on LMS. It can be noted that identification of cultural variables, along with other possible variables, plays a critical role in the outcomes of LMS usage. To overcome cultural barriers, Parrish and Linder-VanBerschot ( 2010 ) investigated the influence of cultural dimensions in online learning, which involved social relationships, epistemological beliefs, and temporal perceptions. Using the cultural dimensions of learning framework as a diagnostic tool, possible solutions to overcome the challenges of multicultural learning were suggested as increased awareness, culturally sensitive communication, modified instructional design processes, and efforts to accommodate critical cultural differences.

Gender factor

Apart from cultural differences, another important factor drawn from the literature review conclusions is the gender difference. Previous studies have reported different characteristics of male and female students involved in online learning. According to a study conducted by Cuadrado-García, Ruiz-Molina, and Montoro-Pons ( 2010 ), male and female students showed significant differences in the assessment and use of e-learning activities. This study involved a bilingual e-learning project between two European universities. The study revealed that female students achieved better final grades than male students with significantly higher resource views on LMS. Furthermore, male students needed more assistance with the online software. These results also support the argument from Bruestle et al. ( 2009 ) who stated that e-learning favors female students due to its flexible and interactive learning approach. Gender difference is also evident in general internet usage patterns (Lim & Meier, 2011 ). Out of four general internet use reasons; ‘social networking’, ‘personal knowledge’, ‘formal learning’, and ‘entertainment’, the males focused more on entertainment whereas the females were engaged with social networking. This corresponds to the study results obtained by Adamus, Kerres, Getto, and Engelhardt ( 2009 ). Their study indicated that female university students focused more on communication and cooperation with openness to other’s proposals. After all, such characteristics of female students highly influenced their learning outcomes in an online training program.

On the other hand, the impact of gender difference was questioned by several studies. Astleitner and Steinberg ( 2005 ) reported that gender differences in web-based learning were insignificant. This study conducted a meta-analysis of fourteen empirical studies related to web-based learning. One of the possible explanations for such results can be that certain features of web-based learning might decrease the gender gaps in cognitive process of information. Another explanation provided in their study was that gender differences are only observed when strong accumulating effects are given during the learning process. In addition, Al-Azawei ( 2019 ) only discovered slight gender differences in LMS acceptance levels. This study involved 302 undergraduate students in Iraq and utilized the extended Technology Acceptance Model (TAM) to predict learners’ perceptions towards LMS adoption. It was found that female students were more concerned with ease of use, whereas male students were more concerned with technology usefulness. However, the differences were not significant. Thus, continuous research on gender effects should be carried out for clarification.

Age and school year factors

Although cultural factors and gender differences were suggested to play an important role in online learning and LMS implementation, previous studies have also shown that students’ age can influence the learning outcomes. According to a study conducted by McSporran and Young ( 2001 ) in a first-year introductory course for the computing systems bachelor degree, older students are more motivated to learn, better at communicating online, and at organizing their learning schedules. In the same study, female students showed better performance than male students did. Hence, both gender and age factors were related to students’ learning outcomes.

As drawn from the literature review, four major factors could affect the dynamic process of smart learning through the operation of LMS. In addition, LMS requires students’ active participation and engagement with learning because they often need to access online course materials without simultaneous prompting or instructions (Beer, Clark, & Jones, 2010 ). This implies that students who largely depend on substantial instructor direction may struggle with LMS, as it demands a certain level of self-discipline (Douglas & Alemanne, 2007 ). You ( 2016 ) also verified the importance of self-regulated learning in online courses based on LMS data measures from 530 college students. Thus, careful examinations of each influencing factor on LMS should be carried out in order to induce self-regulated learning and maximize the learning outcomes. As such, our study focuses on four important variables; ‘cultural dimension’ along with ‘gender’, ‘age’, and ‘school year’. By analyzing students’ preferred LMS functions or design, in relation to these four variables, the study results will extend current understanding of online learner preferences and provide useful guidance for smart learning environments, facilitated by LMS.

Study sample and context

Fraenkel, Wallen, and Hyun ( 2012 ) have categorized case studies as intrinsic (detailed description of one context), instrumental (focusing on a case for comprehending a more detailed phenomenon) and collective (several different or similar cases scrutinized simultaneously), which may include “…one individual, classroom, school, or program…” (p. 435). Based on that description, this study falls under the definition of instrumental case study where the researchers have had an interest in understanding more than how some students value the importance of certain tools/elements for learning management system design. The researchers have been interested in a larger goal of understanding the role of the cultural dimension for learning management system design. Hence, the researchers who conducted this case study; have been more interested in revealing conclusions that could be implemented beyond a particular case than it is. Thus, this study aims to check the following hypotheses which were derived from the conclusions of the profound literature review:

There is a statistically significant difference on each item of learning management design survey with respect to students’ cultural background on being German versus Spanish.

There is a statistically significant difference on each item of learning management design survey with respect to students’ gender.

There is a statistically significant difference on each item of learning management design survey with respect to students’ age.

There is a statistically significant difference on each item of learning management design survey with respect to students’ school year.

The participants of this case study were comprised of German ( n  = 83) and Spanish (n = 83) university students who were attending a fully online ‘Management of Information Systems (MIS)’ course provided by a South Korean university in Fall semester of 2018. Both German and Spanish students utilized the same LMS which was delivered via the university in South Korea. The course took fifteen weeks and one of the researchers of this study was the course instructor of the online course. This course introduces students the basics of modern management information systems and how they have become an integral part of the global operations of the digital companies. The course begins with discussions on the potential of information systems and technologies in improving operational efficiency of common business processes and how they could be managed effectively. Information technology infrastructure, databases and telecommunications are covered earlier than digging deeper into enterprise, supplier and customer applications. Having gained some knowledge of a variety of MIS applications, the students are equipped with practical skills for selection, acquisition and deployment of different information systems, for which the course uses several case studies and exercises.

At the time of data collection, the students were already using the same Korean university LMS together for a month and these students used different LMS (similar features) in their educational lives previously. Additionally, Learning Management Systems (LMS) was one of the management information system applications within the course topics. Hence, the researchers safely assumed that the participants of these study have had enough knowledge on making personal judgement on the survey items.

At the end of week four, the students were given the prepared survey as a voluntary activity to be filled before week four. As Table  1 shows, majority of the participants (66%) are in 18–25 age group and there is nearly an equal gender representation (53% male and 47% female) for the participants.

Although there was an equal number of students for each country ( n  = 83 for both Germany and Spain), there was diversity in school year (since the course was open for the registration to all school levels/years). Table  2 demonstrates that dominant groups were either freshman students (27%) or master students (28%).

Study instrument

The study instrument consists of four demographics related questions which were drawn from the literature review conclusions; gender (male or female), age (under 18, 18–25 or above 25), country (Germany or Spain) and school year (freshman, sophomore, junior, senior or master student). Afterwards, the researchers utilized the learning management design criteria of course textbook written by Laudon and Laudon ( 2018 ) to their survey (Table  3 ). Each category was marked on a five-point Likert scale from ‘not important at all’ to ‘very important’ where the students were grading the importance of each element for a learning management system. When the survey had been finalized, it was uploaded to a survey webpage and kept online for a week. The online survey was five webpage long where the first page was about students’ four demographics and the rest was dedicated to each dimension in Table 3 separately.

Data analysis

The researchers conducted two crosstab analyses on the final dataset; ‘age versus gender’ and ‘school year versus country’. Afterwards, the researchers applied normality test to the dataset on SPSS. Twenty items were checked against four demographic variables whether they show normal distribution on their levels. Both the Kolmogorov–Smirnov and Shapiro–Wilk tests rejected the null hypothesis of a normal population distribution for all four demographics ( p  = 0.05) (Denis, 2019 ). Therefore, the researchers decided to continue with non-parametric statistical techniques.

Additionally, the dataset was checked for its reliability. Twenty items were checked with one hundred sixty six participants and Cronbach alpha was calculated as 0.84, which shows a good score for reliability. Since Cronbach’s alpha does not assume normality (Sheng & Sheng, 2012 ), there was no issue of use for that not-normally distributing dataset.

The researchers calculated the mean scores and standard deviations for twenty items in total, for German students only and for Spanish students only. The results were tabulated and commented accordingly. After these fundamental statistics, the comparison tests were conducted. First of all, three variables (age, gender and country) were checked for each of these twenty items by using Mann Whitney U tests. The significant items were reported with their comments. Lastly, Kruskal Wallis H tests were conducted on twenty items for school year variable with its five levels; freshman, sophomore, junior, senior and master students. The significant items were reported and mentioned accordingly.

The total mean scores and standard deviations of each of the twenty design items were presented in Table  4 ( n  = 166). Additionally, the mean scores and standard deviations were calculated separately for German students ( n  = 83) and Spanish students (n = 83) and tabulated in Table  4 . Within the ‘content management’ dimension, the highest means were observed for private storage (M = 3.82) and online whiteboard (M = 3.59). Integrated offline mode (M = 3.66) and calendar integration (M = 3.59) were revealed as the most valued items of ‘ease of use’ dimension. The dimension ‘communication within the LMS’ was calculated over 3.00 for each of its items; respectively chat system (M = 3.54), notifications (M = 3.42), discussion forum (M = 3.33) and survey feature (M = 3.24). For the last dimension of screen ‘design’, the participants mostly valued marking files/courses as their favorites (M = 3.82) and choosing a personal layout (M = 3.78).

When the mean scores of each country were considered, it is easy to see that many design items were valued differently. Thirteen items were more valued by Spanish students and seven items were more valued by German students. The higher mean score for each item was highlighted grey in Table  4 . The simple mean score differences may not show the real statistically differentiating items. Therefore, the comparison tests of Mann-Whitney U tests and Kruskal Wallis H tests were run for better understanding.

First Mann-Whitney U tests were conducted for gender variable (male versus female) on twenty items of learning management system design. The only significantly differentiating item was appeared on the first item of ‘ease of use’ dimension; ‘allowing downloading multiple files’ ( U  = 2811.500, p  = 0.037). The mean rank demonstrated that male students (mean rank = 90.55, n  = 88) valued the downloading multiple files feature more than female students do (mean rank = 75.54, n  = 78).

Other Mann-Whitney U tests were implemented for age variable (18–25 versus above 25) on twenty learning management system design items. The results yielded only one single significantly differentiating item which belongs to ‘communication within the LMS’ dimension; ‘survey feature’ ( U  = 2450.000, p  = 0.023). The mean rank for ‘above 25’ group ( n  = 56) is higher than ‘18–25’ age group ( n  = 110); 94.75 and 77.77 respectively.

The last Mann-Whitney U tests were run for the country variable (Germany versus Spanish). As Table  5 demonstrates, sixteen items were significantly differentiated around country variable. Among these sixteen significantly differentiating items, only four design items’ mean ranks were higher for German students; ‘uploading assignments’, ‘accessing learning materials’, ‘learning materials are available before lectures’ and ‘simple navigation structure’. The Spanish students’ mean ranks were higher than German students for the other twelve design items; ‘comment feature’, ‘online whiteboard’, ‘private storage’, ‘easy enrollment of subject’, ‘integrated offline mod’, ‘calendar integration’, ‘chat system’, ‘discussion forum’, ‘survey feature’, ‘notifications’, ‘choose a personal design/layout’, and ‘mark files/courses as favorite’.

The last comparison tests were conducted on the school year variable for twenty design items separately. The Kruskal Wallis H tests results unfolded eleven significantly differentiating design items around the school year variable; ‘comment feature’, ‘online whiteboard’, ‘private storage’, ‘allowing downloading multiple files’, ‘learning materials are available before lectures’, ‘calendar integration’, ‘discussion forum’, ‘survey feature’, ‘notifications’, ‘choose a personal design/layout’ and ‘mark files/courses as favorite’. Although the number of students in each school level differs from each other, the mean ranks could still be used to get a deeper understanding for school years on each design item. Table  6 shows that except ‘learning materials are available before lectures’ design items where the master students had the highest mean rank, freshman students’ mean ranks were the highest for the other ten significantly differentiating design items.

Discussion and conclusion

As LMS has become a crucial element of different instructional contexts, the efforts trying to unfold its successful design factors have been studied more than ever before. The previous studies enlisted four essential success factors for LMS implementations. Therefore, this study aims to understand LMS design from a cultural point of view in additional variables of gender, age and school year. The general results clearly demonstrated that one unique LMS design will not be useful and appreciated by the students all the time. In that sense, other than setting up a commonly designed LMS on their school smart systems, the managers/instructors should prefer a more user centered approach where the LMS will be tailored according to their students’ demographics (especially the variables discussed in this study).

When German and Spanish students were compared with non-parametric statistical tests, it seemed that Spanish students generally more valued various features of LMS. In particular, Spanish students claimed that ease of use and communication within the LMS are important features for their learning. In the content management section, Spanish students also valued comment feature and online whiteboard as evident in the mean scores and the Mann-Whitney U test results. This implies that Spanish students would prefer learning through communication. Hence, the instructional designers or practitioners should offer more interactive and communicative opportunities to Spanish students on their LMS.

On the other hand, German students have put a strong emphasis on LMS features such as ‘uploading assignments’, ‘accessing learning materials’, ‘learning materials are available before lectures’ and ‘simple navigation structure’. Most of these features are directly related to the final grade and individual learning. Such different characteristics of German and Spanish students could affect their learning behaviors in a way that German students would value goal-oriented individual learning and Spanish students would value process-oriented group learning with active communication. This gives clues to the instructors while designing their instructional activities on LMS. For instance, Spanish students should be directed toward more group assignments whereas German students would appreciate more individual self-studies and exercises.

In fact, this study results are relevant to a four-dimensional model of cultural differences proposed by Hofstede ( 1986 ). Based on Hofstede’s model, the individualism-collectivism dimension provides a possible explanation for different characteristics of German and Spanish students. As described by Mercado et al. ( 2004 ), individualism values personal achievement or well-being of an individual, which suits with the characteristics observed from German students. On the other hand, collectivism highlights group achievement or group actions, which can match with the characteristics shown by Spanish students. These findings can also explain both countries’ different cultural dimension scores on individualism versus collectivism as mentioned in the literature review (Hofstede et al., 2010 ). As German students highly valued two LMS features, ‘accessing learning materials’ and ‘learning materials are available before lectures’, it implies that German students want to be prepared for their classes. Such preparation might be related to uncertainty avoidance for what they will learn in class. If so, it will contradict the results obtained by Hofstede et al. ( 2010 ) as Germany’s uncertainty avoidance score was lower than Spain. Therefore, the fundamental reason for accessing learning materials should be clarified to further explain such contradictory results. The remaining two cultural dimensions, power distance and masculinity, could not be related to our study results as students’ perspectives on hierarchical learning and competition-based learning were not assessed. Despite the cultural differences, it should be also noted that both groups of students similarly valued certain features of LMS such as reviewing grade, downloading multiple files, language selection, and access through mobile application. In both cultures, the LMS features related to users’ convenience seem equally important.

In this study, significant gender differences were not observed. The only difference observed was that male students valued downloading multiple files feature of LMS more than female students, which could mean male students favor efficiency when using LMS. As suggested by Astleitner and Steinberg ( 2005 ), LMS features might actually reduce gender differences compared to the offline class environments or there were not enough accumulating effects to induce gender differences in our study. Another possible explanation could be that gender differences are created due to the learning materials or course contents rather than LMS itself. Further elucidation on gender effect is required in future studies.

In terms of the age variable, this study results indicated that higher age group students more valued communication within the LMS, in particular, survey feature. This result is also supported by McSporran and Young ( 2001 ) as their study showed better communication skills from older students. As a learner’s age increases, it might also develop online/offline communication skills and thus learning through communication becomes a preferable option. However, it should be noted that communication features of LMS were not necessarily valued by students with the higher school year. In other words, the school year variable does not induce the same effect on the learning process or learning preference as the age variable does. Therefore, instructors should not assume similar learning behaviors between the higher age group and higher school year group when designing and implementing an LMS.

When the school year variable was examined, numerous features of LMS were highly valued by the freshman students. One of the possible reasons would be an exposure to new smart learning environments. As freshman students need to adapt in the university education system, they need to pay a particular attention to each element of an LMS. Once the adaptation period is over, the significance of LMS features might be reduced and students will gradually utilize specific LMS functions that are directly relevant to their learning process. Indeed, each feature of LMS was valued differently in each school year apart from the freshman period. It is interesting to note that the master students highly valued availability of the learning materials before lectures. This possibly indicates that postgraduate programs emphasize more on pre-class learning, which is often observed in learner-centered environments.

Our study explored various features of LMS valued by different groups of students based on their cultural background, gender, age, and school year. Out of the four hypotheses tested in our study, the first (cultural background) and the fourth (school year) hypotheses were validated whereas the second (gender) and the third (age) hypotheses were partially validated. Although not every hypothesis was fully validated, there are several important recommendations for instructors or education providers based on our study results. First of all, it is advised that future LMS design should consider the four-dimensional model of Hofstede ( 1986 ), especially the individualism-collectivism dimension to cater for various learning needs of international students. Understanding the effect of culture on LMS design, delivery and implementation will provide more user satisfaction leading toward more success stories in education. Secondly, learning materials on LMS should be checked for possible inducing factors of gender differences. In that sense, the instructors should be informed about gender bias issues. Thirdly, more communication features of LMS will be effective in the courses with higher age groups. Lastly, LMS can provide more guidelines or assistance for freshman students and create a wide range of learner-centered environments for postgraduate programs.

Since this study was delimited to two specific cultures, prospective studies must focus on adding more variety to similar culture based design studies. In order to gain a further understanding of LMS and smart learning process, future studies should investigate more various cultural groups and their learning characteristics. Moreover, due to the sample limitation of this study, the researchers highly recommend to conduct prospective studies with larger sample size to analyze group with parametric techniques. If possible and available, students’ LMS logs (including the most commonly used tools) should be analyzed for a better understanding of LMS tools and their usage by different cultures. Similarly, different variables examined in this study could be compared with assessment or examination grades to identify which LMS features can maximize the learning outcomes for a particular group of students. Additionally, qualitative interview schedules should be integrated into culture based studies to understand its effects in depth.

Since this study implemented the convenience sampling which might have the disadvantage of bias, the similar studies should be replicated in different courses or universities to check if the observed results are due to onetime occurrence. Moreover, LMS has also been utilized in business world where different companies' training activities are supported by these smart systems. In that sense, the research in business world could assist us to understand the deeper influence of culture.

The instructional stakeholders must always remember that future studies on culturally sensitive LMS design will contribute to the achievement of better learning in the waves of upcoming digital revolution era.

Availability of data and materials

The corresponding author declared here all types of data used in this study available for any clarification. The author of this manuscript ready for any justification regarding the data set. To make publically available of the data used in this study, the seeker must send an email to the mentioned email address. The profile of the respondents was completely confidential.

Abbreviations

• Learning management systems

Master of business administration

Management of information systems

Adamus, T., Kerres, M., Getto, B., & Engelhardt, N. (2009). Gender and e-tutoring - A concept for gender sensitive e-tutor training programs. In Fifth European symposium on gender & ICT digital cultures: Participation - empowerment - diversity . University of Bremen March 5-7, 2009 - Retrieved from http://www.informatik.uni-bremen.de/soteg/gict2009/proceedings/GICT2009_Adamus.pdf .

Al-Azawei, A. (2019). The moderating effect of gender differences on learning management system acceptance: A multi-group analysis. Italian Journal of Educational Technology , 27 (3), 257–278. https://doi.org/10.17471/2499-4324/1088 .

Article   Google Scholar  

Astleitner, H., & Steinberg, R. (2005). Are there gender differences in web-based learning? An integrated model and related effect sizes. Association for the Advancement of Computing in Education , 13 (1), 47–63.

Google Scholar  

Asunka, S. (2016). Helping faculty overcome cultural barriers to adoption and use of web-based learning technologies: A participatory action research approach. In Revolutionizing education through web-based instruction , (pp. 300–316).

Chapter   Google Scholar  

Beer, C., Clark, K., & Jones, D. (2010). Indicators of engagement . In Proceedings Ascilite Sydney 2010 , (pp. 75–86) Retrieved from http://ascilite.org.au/conferences/sydney10/procs/Beer-full.pdf .

Bove, L., & A., & Conklin, S. (2020). Learning strategies for faculty during a learning management system migration. Online Journal of Distance Learning Administration , 23 (1), 1–10.

Bruestle, P., Haubner, D., Schinzel, B., Holthaus, M., Remmele, B., Schirmer, D., & Reips, U. D. (2009). Doing e-learning/doing gender? Examining the relationship between students' gender concepts and e-learning technology. In Fifth European symposium on gender & ICT digital cultures: Participation - empowerment - diversity . University of Bremen March 5–7, 2009. Retrieved from http://www.informatik.uni-bremen.de/soteg/gict2009/proceedings/GICT2009_Adamus.pdf .

Cuadrado-García, M., Ruiz-Molina, M. E., & Montoro-Pons, J. D. (2010). Are there gender differences in e-learning use and assessment? Evidence from an interuniversity online project in Europe. Procedia - Social and Behavioral Sciences , 2 (2), 367–371. https://doi.org/10.1016/j.sbspro.2010.03.027 .

Denis, D. J. (2019). SPSS data analysis for univariate, bivariate, and multivariate statistics . Hoboken: Wiley.

Douglas, I., & Alemanne, N. D. (2007). Measuring student participation and effort . Algarve: Paper presented at the international conference on cognition and exploratory learning in digital age Retrieved from https://www.researchgate.net/publication/241134504_Measuring_student_participation_and_effort .

Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2012). How to design and evaluate research in education , (8th ed., ). New York: McGraw-Hill Humanities/Social Sciences/Languages.

Hofstede, G. (1986). Cultural differences in teaching and learning. International Journal of Intercultural Relations , 10 (3), 301–320. https://doi.org/10.1016/0147-1767(86)90015-5 .

Hofstede, G., Hofstede, G. J., & Minkov, M. (2010). Cultures and organizations: Software of the mind. Revised and expanded , (3rd ed., ). New York: McGraw-Hill.

Hunt, A., & Tickner, S. (2015). Cultural dimensions of learning in online teacher education courses. Journal of Open, Flexible, and Distance Learning , 19 (2), 25–47.

Laudon, K. C., & Laudon, J. P. (2018). Management information systems: Managing the digital firm , (15th ed., ). New York: Pearson.

Lim, K., & Meier, E. B. (2011). Different but similar: Computer use patterns between young Korean males and females. Educational Technology Research and Development , 59 (4), 575–592. https://doi.org/10.1007/s11423-011-9206-5 .

Liu, X., Liu, S., Lee, S. H., & Magjuka, R. J. (2010). Cultural differences in online learning: International student perceptions. Journal of Educational Technology & Society , 13 (3), 177–188.

McSporran, M., & Young, S. (2001). Does gender matter in online learning? Unitec New Zealand working paper. Retrieved from https://repository.alt.ac.uk/348/1/ALT_J_Vol9_No2_2001_Does%20gender%20matter%20in%20online%20l.pdf .

Mercado, S., Parboteeah, K. P., & Zhao, Y. (2004). On-line course design and delivery: Cross-national considerations. Strategic Change , 13 (4), 183–192. https://doi.org/10.1002/jsc.677 .

Parrish, P., & Linder-VanBerschot, J. A. (2010). Cultural dimensions of learning: Addressing the challenges of multicultural instruction. International Review of Research in Open and Distance Learning , 11 (2), 1–19. https://doi.org/10.19173/irrodl.v11i2.809 .

Sheng, Y., & Sheng, Z. (2012). Is coefficient alpha robust to non-normal data? Frontiers in Psychology , 3 (34). https://doi.org/10.3389/fpsyg.2012.00034 .

Swierczek, F. W., & Bechter, C. (2010). Cultural features of e-learning: A euro Asian case study. In J. M. Spector et al. (Eds.), Learning and instruction in the digital age , (pp. 291–308). New York: Springer.

Tarhini, A., Hone, K., Liu, X., & Tarhini, T. (2017). Examining the moderating effect of individual-level cultural values on users’ acceptance of e-learning in developing countries: A structural equation modeling of an extended technology acceptance model. Interactive Learning Environments , 25 (3), 306–328. https://doi.org/10.1080/10494820.2015.1122635 .

Yakubu, M. N. (2019). The effect of quality antecedents on the acceptance of learning management systems: A case of two private universities in Nigeria. International Journal of Education and Development using Information and Communication Technology , 15 (4), 101–115.

You, J. W. (2016). Identifying significant indicators using LMS data to predict course achievement in online learning. The Internet and Higher Education , 29 , 23–30.

Download references

Acknowledgements

Not applicable.

Authors declare that no funding for the research was provided from any side.

Author information

Authors and affiliations.

Technology Studies, Endicott College of International Studies, Woosong University, Daejeon, South Korea

Hasan Tinmaz

Global Healthcare Management, Sol International School, Woosong University, 171 Dongdaejeon-ro, Dong-gu, Daejeon, 34606, South Korea

Jin Hwa Lee

You can also search for this author in PubMed   Google Scholar

Contributions

Hasan Tinmaz was responsible from method and results sections. Jin Hwa Lee was in charge of literature review and discussion sections. The authors worked together on the rest of the manuscript. The authors read and approved the final manuscript.

Corresponding author

Correspondence to Jin Hwa Lee .

Ethics declarations

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Tinmaz, H., Lee, J.H. An analysis of users’ preferences on learning management systems: a case on German versus Spanish students. Smart Learn. Environ. 7 , 30 (2020). https://doi.org/10.1186/s40561-020-00141-8

Download citation

Received : 13 April 2020

Accepted : 30 September 2020

Published : 14 October 2020

DOI : https://doi.org/10.1186/s40561-020-00141-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Smart learning

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Springer Nature - PMC COVID-19 Collection
• PMC10248339

A systematic review on factors influencing learning management system usage in Arab gulf countries

Twana tahseen sulaiman.

Department of Business Administration, College of Administration and Financial Sciences, University of Cihan-Erbil, Erbil, Kurdistan Region 44001 Iraq

Associated Data

Not applicable.

Although the successful implementation of the Learning Management System (LMS) in most of the universities in the Arab Gulf Countries (AGC), little consideration has been paid to exploring LMS usage. This paper provides a systematic review of the current literature focusing on the most critical factors influencing LMS usage in AGC. The extant literature was identified through six electronic databases from 2013 to 2023. Academic articles were reviewed if they contained a relevant discussion of the factors influencing LMS acceptance and adoption conducted in AGC. Results from a systematic review of 34 studies showed that 15 studies were centred in Saudi Arabia. The results also, revealed that Technology Acceptance Model was the dominant model employed, and students were the main subject of studies. Moreover, the quantitative approach was the preferred design. Overall, forty-one factors were identified, and the results show that the following eight factors appear most frequently: Perceived Ease of Use, Perceived Usefulness, Social Influence, Performance Expectancy, Effort Expectancy, Facilitating Conditions, Self-efficacy, and Attitude. This review will be valuable for future research and helpful for higher education decision-makers who intend to use eLearning to overcome the challenges they face in using LMS effectively.

Introduction

The learning management system has proven to be an effective alternative to traditional classroom instruction, which has allowed educational programs at the vast majority of universities to continue operating normally despite the COVID-19 pandemic (Mailizar, et al., 2021 ). LMS presents educational institutions with a unique opportunity to innovate with respect to the delivery of conventional pedagogical practices due to its numerous advantages and benefits (Cao, et al., 2022 ). However, the Arab Gulf Countries present a challenge when it comes to the incorporation of technology into the educational systems of their countries (Alsswey, et al., 2020 ). This may result from several factors, such as those related to technology, culture, society, and the role of the instructor, which may inhibit the adoption of eLearning among lecturers. Besides, the speed of internet connections is increasing across all of the Arab Gulf Countries, including Saudi Arabia, Kuwait, Oman, Qatar, Iraq, and the United Arab Emirates (Weber & Hamlaoui, 2018 ). This is one of the most critical factors in the expansion of eLearning, which is a form of distance education. Although LMS is a powerful platform used by almost all the universities in developed countries (Phan, et al., 2022 ), the level of LMS usage is still low in AGC (Alsswey, et al., 2020 ). In order to increase the level of eLearning usage, countries in the Arab Gulf should understand the relevant models and theories of LMS adoption. Because of this, this study aims to offer a more in-depth comprehension of the implementation of LMS in these countries.

A view appointed by Alsswey et al. ( 2020 ) states that increasing demand for higher education in AGC cannot be met solely through traditional face-to-face learning delivery; therefore, it is advantageous to use modern approaches such as eLearning, blended and online learning, which are supported by LMSs. Further, in the view of Kunene and Maphosa ( 2020 ), LMS enables universities to better manage users, courses, and instructors with testing capabilities and to facilitate the generation of student transcripts, reports, and activity notifications. LMS can accelerate the teaching and learning process and enhance communication between users at all times and locations (Sinclair & Aho, 2018 ).

Despite the growing number of systematic reviews papers examining LMS adoption worldwide (Bervell & Umar, 2017 ; Gamage, et al., 2022 ; Granić & Marangunić, 2019 ; Ziraba, et al., 2020 ), none of these have exclusively investigated the utilization of LMS in AGCs. The rationale for conducting this study is to address this research gap. The aim is to enhance the current literature by providing a comprehensive overview of the most recent LMS research publications in AGC.

A key factor observed by prior studies has to do with the initial acceptance by potential lecturers and students who are to use it for pedagogical purposes. According to Alsswey et al. ( 2020 ), rejection rates are also high despite the advantages brought about by LMS. This development has increased awareness in the Arab context of the fact that LMS usage is still a novelty in the AGC. Diverse research findings have revealed dimensions of factors influencing LMS acceptance; however, no systematic review studies provide a comprehensive view of the diverse LMS adoption and use research conducted in the AGC. This provides a rationale for the need to collect these studies from the last ten years to establish a distinct focus on LMS adoption and use in AGC to establish trends for future research. This systematic review paper seeks to fill the gap, by answering the following research questions:

• Which countries have contributed to LMS studies within AGC?
• Which models or theories have been used to study LMS adoption and use in AGC?
• What methodologies have been employed in studying LMS adoption and use in AGC?
• What are the critical factors influencing the adoption and use of LMS in AGC?

This study has the potential to contribute to the literature by systematically reviewing research on LMS adoption and use across the AGC and by providing more comprehensive evidence on the critical factors encountered in promoting LMS usage. In addition, it provides recommendations for future research areas. The report concludes with pertinent policy and practice recommendations for AGC technology integration based on the findings of all the reviewed studies. This paper is divided into several sections to present the study’s findings in a clear and organized manner. The next section will summarize the previous studies and highlight recent studies. The third section will describe the methodology used for the review, including the scope of the study, the search strategy employed to identify relevant literature, and the methods used to analyze the data. The fourth section will present the study’s empirical results, while the fifth section will discuss the implications of the findings. Finally, the paper will conclude with a summary of the essential findings and their significance in Sect.  6 .

Previous studies

Although numerous systematic review papers have been published on LMS adoption worldwide, no published systematic reviews exist on LMS adoption in AGC. Among the existing systematic reviews related to AGC, only two have focused on mobile learning adoption. Alsswey et al. ( 2020 ) reviewed 31 publications to investigate the current evidence on the use of mobile learning in AGC among instructors and students. Their findings indicate that students’ and instructors’ acceptance and utilization of mobile learning are the most significant issues. Moreover, the study revealed that the lack of research on leadership and policy practices in AGC may lead to the failure of technology adoption. Similarly, Alsswey and Al-Samarraie ( 2019 ) systematically reviewed 24 research articles published between 2008 and 2018. The study revealed that the adoption of mobile learning in AGC is influenced by various factors, such as technological, educational, cultural, organizational, and individual factors. The study also identified several obstacles to adopting mobile learning in AGC. Therefore, the present study will review articles on LMS utilization among AGC.

In the early years of this century, an increasing number of universities in AGC have implemented learning management systems. However, research studies on LMS adoption did not emerge until 2010, when educators began recognising the advantages of LMS utilization. Al-Busaidi and Al-Shihi ( 2010 ) found that perceived usefulness and ease of use were significant predictors of LMS adoption among instructors, whereas subjective norms and voluntariness did not significantly influence LMS adoption. Furthermore, various studies have examined the factors influencing AGC adoption and utilization of LMS in higher education contexts. For instance, Alkharang and Ghinea ( 2013 ) investigated the factors influencing the adoption of eLearning in Kuwait and found that facilitating conditions and behavioural intentions influenced users’ eLearning use. Mouakket and Bettayeb ( 2015 ) studied the factors influencing the use of LMS, particularly by university instructors, and identified that perceived usefulness, instructors’ intentions, user interface design, technical support, and training influenced Blackboard system satisfaction. Alharbi and Drew ( 2014 ) conducted research in Saudi Arabia and identified that perceived usefulness, ease of use, LMS availability, prior LMS experience, and job relevance were significant predictors of LMS adoption. Kurdi et al. ( 2020 ) presented an empirical investigation into the factors influencing the acceptance of eLearning by university students in the United Arab Emirates. The findings indicated that eLearning computer self-efficacy, social influence, enjoyment, system interactivity, computer anxiety, technical support, perceived usefulness, perceived ease of use, and attitude were the most significant predictors of behavioural intention to use eLearning.

Recent studies have identified several factors that influence the adoption and utilization of LMS in AGC. Sulaiman et al. ( 2023 ) found that perceived usefulness and perceived ease of use mediated the relationship between actual usage and predictors of LMS adoption in Iraq. Service quality, service quality, and government policy were also found to be significant predictors of LMS adoption in Iraq. Fattah et al. ( 2022 ) utilized the TOE framework to investigate LMS usage in Iraq and identified relative advantage, technical compatibility, education institutes sizes, top management support, and LMS knowledge as significant predictors of LMS adoption. Similarly, Altalbe ( 2021 ) investigated the moderating effect of instructor support on eLearning systems’ utilisation in higher education during the COVID-19 pandemic in Saudi Arabia. The results indicated that the quality factors significantly influenced students’ actual utilization more than the usability and interaction factors, which have weaker influences on actual usage. Moreover, Al-Mamary ( 2022 ) explored the factors influencing undergraduate students’ LMS use in Saudi Arabian universities. The study adopted UTAUT as the theoretical framework to investigate the primary factors influencing students’ behavioural intention to use LMS. The study found that performance expectations, effort expectations, and social influence significantly impact students’ behavioral intent to use LMS. These studies highlight that technological, cultural, social, and institutional factors play a crucial role in adopting and using LMSs in AGC.

Overall, the available literature underscores the intricacies and diverse nature of factors that affect the adoption and usage of LMS in AGC, which are subject to the technological, cultural, social, and institutional settings. Hence, a systematic review of the extant literature is essential to consolidate the findings of earlier studies and discern the key drivers that impact LMS usage in this region.

Methodology

This paper adopted the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) from Moher et al. ( 2009 ) to answer the research questions. The systematic procedure comprises four phases of PRISMA: identification, screening, eligibility, and inclusion criteria. The number of articles identified through online database searches during the identification phase. After removing duplicates, the researcher determined the total number of articles during the screening phase. In the eligibility phase, the number of articles that were evaluated and excluded because they do not fall within the scope of the study is indicated. Finally, the inclusion stage was conducted to determine how many articles were included in the final analysis.

Search strategy

This systematic review paper searched six databases, including Google Scholar, Science Direct, Emerald, Springer, Scopus, and IEEE, for relevant research on the acceptance of eLearning and LMS in the AGC. Each selected Articles’ reference list was also analyzed in order to collect additional relevant sources. The initial step started with searching for a combination and variation of a set of keywords; “eLearning” OR “Learning Management System” OR ‘LMS’ OR “Moodle” OR “Blackboard” OR “Course Management System” AND “adoption” OR “usage” OR “acceptance” AND “Arab Gulf Countries (including Iraq, Kuwait, Bahrain, Saudi Arabia, Qatar, United Arab Emirate and Oman) OR “higher education institutions” OR “universities”. The terms “adoption”, “usage”, and “acceptance” are used interchangeably in this study because their meanings are so similar. The inclusion and exclusion criteria were then implemented. The researcher utilized Microsoft Excel to store and document the acquired articles.

Eligibility criteria

All retrieved studies pertaining to the LMS that met the inclusion and exclusion criteria were evaluated. This study analyzed articles on AGC lectures and students’ perceptions of LMS usage. Eligible articles had to be written in English and utilize qualitative, quantitative, or mixed methodologies. In addition, the selected articles had to have been published from January 2013 till January 2023 in peer-reviewed journals. Articles focusing solely on the LMS implementation phase, training, and technical aspects were excluded from the study. Protocols, policy briefs, oral presentations, and reports from non-governmental organizations were excluded from the review.

Prior research on AGC discussed in this section revealed that essential factors should be investigated in future studies, which justifies the need for the current investigation (Alammary, et al., 2021 ). On the basis of previous research, this study will provide a novel perspective on identifying the essential factors required to comprehend LMS usage. Therefore, it is initial to identify critical factors that influence the LMS in AGC.

The results of the primary search of the relevant articles totalled 2525 articles without applying any inclusion or exclusion criteria. After removing all the duplicates, the total number of retrieved articles was (2498). It was decided not to include the research after reviewing the titles and abstracts (2153). The full texts of the remaining 372 papers were examined, and a comparison was made to both the inclusion and exclusion criteria. This led to the removal of 317 articles, which was then followed by the removal of 21 articles that did not discuss the factors that influence the adoption and use of LMSs in educational settings. This led to the production of thirty-four articles that met the criteria for inclusion in this study. Appendix A provides a summary of the reviewed articles that were chosen. Figure ​ Figure1 1 summarizes the results of the literature search and screening procedures.

Flow diagram of the selection process for including articles

Empirical results

Countries have contributed to lms studies within agc.

The preliminary results for AGC countries from which studies were conducted, along with the corresponding number of studies, are presented in Table  1 .

Arab Gulf Countries with number of studies

With reference to Table  1 , articles spanned across all countries in the AGC region except Bahrain. With respect to the number of studies, Saudi Arabia had 15, being the highest number of studies, representing 44.12% out of the total number of 34. This was followed by Iraq with 7 studies and the United Arab Emirates with six studies. Three studies were conducted in Oman, followed by Kuwait with two studies. Only one studies conducted in Qatar.

Models or theories have been used to study LMS adoption and use in AGC

In an effort to respond to the second research question, this section examines the various theories and models utilized in acceptance and adoption studies in AGC. Table  2 displays the outcomes.

Models Used in Studies

Highlights from Table  2 indicate that most studies (13) representing 38.23% employed the TAM as the theoretical model suitable for their research. The second model of preference was the UTAUT, being utilized in 7 studies representing 20.59%. This was Followed by UTAUT2 with three studies. Only two studies adopt ECM. Seven out of the 34 studies representing 20.59% did not use any model for their studies. The other models presented in the table, such as TOE and DOI were employed in only one study each.

Methodologies have been employed in studying LMS adoption and use in AGC

The information in Table  3 pertains to the research design and methodology utilized in the various AGC studies.

Methodologies Used in Studies

Essential details from Table  3 demonstrate that the quantitative research design dominated most of the studies. This is underpinned by the fact that 30 out of the total studies representing 88.23% employed this research design. This was followed by the qualitative recording 3 (8.83%). Only one study used a mixed method approach, the least used method. Conversely, the subjects selected for studies are depicted in Table  4 , which provides information on the subject in the 34 articles selected for this study.

Subjects Used in Studies

Details from Table  4 indicate that half of the 34 studies used university students as their subject of study. For the remaining studies, 11 (32.35%) used instructors as a subject of study. Only 6 (17.64%) out of the 34 studies focused on both instructors and students for their research.

Factors influencing adoption and use of LMS in AGC

In an effort to answer the fourth research question of this study, the researchers compiled a list of the various factors reported across the reviewed studies as LMS usage intention determinants. The analysis of these studies yielded 41 factors with their occurrence frequencies. Figure  2 illustrates the outcomes.

Critical factors of LMS usage

With reference to the statistics in Fig.  2 , there is an indication that higher frequencies of 18 19 go for Perceived Ease of Use, which is considered the first important factor, followed by the second important factor, Perceived Usefulness, with 18. The third critical factor were Social Influence, Effort Expediency, Performance Expectancy and Facilitating condition, with 12 each. Self-efficacy was the next important factor, with a frequency of 8, followed by Attitude with 7. Subsequently, technical support and training with a frequency of 4 each. Followed by quality factors such as System Quality, Information Quality and Service Quality which are the last important factor with a frequency of 3 each, followed by Top Management Support with a frequency of 2. Last of all, the remaining factors mentioned in Fig.  2 are repeated only one time.

Discussion of the finding

According to the results of the study, Saudi Arabia had the highest number of LMS acceptance studies. According to the World Bank’s report on country classification by income from 2021, Saudi Arabia was classified as a high-income country. Since 2010, Saudi Arabia, a country with a high per capita income, has invested in the integration of technology into higher education. This led to the rise of eLearning programs in Saudi Arabian higher education institutions, necessitating additional LMS acceptance studies (Asiri, 2012 ). Iraq ranked second in Table  1 after Saudi Arabia. Within the Iraqi context, Jamil ( 2017 ) indicated that there was a trend of institutions of higher education acquiring LMS platforms, particularly Moodle. Sulaiman et al. ( 2019 ) provide a conceptual framework indicating factors that are supposed to be examined in the Iraqi context. Moodle LMS was implemented successfully at the University of Kufa, leading to an increase in eLearning in Iraq (Abdulmohson, et al., 2022 ). This prompted Iraqi researchers to seek out variables that affect LMS utilization.

The United Arab Emirates (UAE) ranked third in Table  1 behind Iraq, which UAE also considered a high-income nation. It has been stated by Daouk and Aldalaien ( 2019 ) that UAE invested heavily in the incorporation of Information and Communication and Technology (ICT) into the learning process. UAE invest heavily in the incorporation of ICT into the learning process (Cao, et al., 2022 ). Oman ranked fourth with three studies according to Table  1 . In the Omani context, Al-Busaidi and Al-Shihi ( 2010 ), is the first article in the AGC published on the LMS adoption to explain the challenges facing LMS acceptance among Omani instructors. This encourages Omani researchers to examine the most critical factors influencing LMS acceptance. This partially explains the justification for why those Arab nations were researched in LMS studies.

The Technology Acceptance Model (TAM) was the most prevalent model employed by researchers in LMS studies in AGC. This was previously supported by Venkatesh and Davis ( 2000 ), who stated that TAM was the most widely used model in research on users’ technology acceptance. However, this review found that most of the studies in AGC that utilized TAM were basically based on the original TAM developed by Davis et al. ( 1989 ). The original TAM positions view perceived ease of use as an independent variable for both perceived Attitude and Perceived Usefulness (Dishaw & Strong, 1999 ). This eliminates the direct effects of perceived usability on intent to use. Subsequently, UTAUT was the second model among reviewed studies in terms of frequency of use. Thus, UTAUT remains unpopular in AGC research despite twenty years of existence. Park ( 2009 ) used TAM to explain how individuals adopt and utilize eLearning or online learning systems.

Alternatively, UTAUT and UTAUT2 could be more valuable since they have all four variables (Performance Expectancy, Effort Expectancy, Social Influence and Facilitating Conditions). Recent research has demonstrated that determining Behaviors Intention outperforms all other technology acceptance models (Venkatesh, et al., 2003 ). Using moderators within the UTAUT and UTAUT2 models, regionally specific contextual factors could be tested for the occurrence of differences in intentions. As indicated by Venkatesh and Bala ( 2008 ), cultural differences are crucial in determining technology use intentions. Therefore, additional research with UTAUT and UTAUT2 is required in AGC to confirm this claim.

Regarding the research design, the majority of reviewed studies employed a quantitative method (30 out of 34). A limitation of the quantitative approach is the omission of narrative details that could be useful for enhancing the quantitative analysis findings (Creswell, 2012 ). Although a quantitative approach is rigorous, the addition of a qualitative component will make it more effective. According to Creswell ( 2012 ), in recent years, mixed method procedures, which combine quantitative and qualitative techniques, have gained in popularity. Mixing quantitative and qualitative research methods has the potential to benefit the majority of research projects (Chen & Hirschheim, 2004 ). In information systems research, Venkatesh et al. ( 2012 ) argue that there is a need to close the gap between quantitative and qualitative data. The advantages of mixed methods research design provide a superior method for explaining and comprehending the complexities of organizational and social phenomena, particularly in terms of technology acceptance (Venkatesh, et al., 2012 ). Thus, it be incumbent on information system researchers in AGC to utilize mixed methods to conduct research. This will facilitate the use of structured and open-ended questionnaires or interview guides in the data collection process to compensate for weaknesses that may arise when only one instrument type is employed (Creswell & Creswell, 2018 ).

In terms of subject of the study, 50% of the studies utilized students as research subjects, as evidenced by the review’s findings. There were only 11 studies that focused on instructors alone. In recent literature, the significance of instructors’ technology adoption beliefs has been highlighted. For instance, Tondeur et al. ( 2017 ) suggest that to support the complex interrelationships between instructors, students, and the educational setting as a whole, it is necessary to comprehend instructors’ views on technology. This stance was earlier supported by Cheng et al. ( 2021 ), who stated that it would be very beneficial for research to be more greatly and explicitly focused on instructors’ beliefs. This context is where instructors typically extend their influences, shaping students’ beliefs. This is especially crucial when addressing educational reform issues such as using LMS for instruction. In addition, some scholars have criticized the use of students as research subjects, specifically in LMS acceptance studies (Alshehri & Alahmari, 2021 ; Alturise, 2022 ; Mohamed Riyath & Muhammed Rijah, 2022 ). Suzanne et al. ( 2019 ) criticized the use of students as subjects in LMS technology acceptance research, arguing that results cannot be generalized to the real world because students are primarily motivated by grades and other factors. On the other hand, instructors unaffected by grades or favour are more likely to identify genuine obstacles to LMS usage (Al-Busaidi & Al-Shihi, 2010 ). When instructors have a favourable view of technology integration, they are likely to influence students to use it positively and vice versa (Gautreau, 2011 ). As instructors are direct implementers of technology in the teaching and learning process and can serve as quick guides and role models for students, focusing on their usage intentions becomes necessary.

This study revealed that Perceived Ease of Use and Perceived Usefulness were the most influential factors of LMS utilization in order of priority and significance. The Perceived Ease of Use and Perceived Usefulness of a learning management system by instructors and students are positively correlated with their frequency of use. When instructors and students perceive that LMS usage will be simple and beneficial for them, they are more likely to value it, which increases their desire to use it (Alshammari, 2020 ). Their perceptions of usefulness lead them to realize that the use of LMS will have positive effects on teaching and learning.

Four UTAUT related factors, namely Social influence, Performance Expectation, Effort Expectancy, and Facilitating Conditions, are considered the second most influential factors on LMS use and adoption. According to Venkatesh et al. ( 2003 ), When potential adopters of an information system believe that using the system will result in a promotion, salary increase, or increase in output gains, their intention behaviour is positively affected. Social influence reflects the impact of other people’s (peers, instructors, and friends) beliefs on the Intention or use behaviour of individuals (Venkatesh, et al., 2003 ). Alshammari et al. ( 2016 ) reported that employees are socially influenced by the beliefs of their peers regarding eLearning, which in turn affects their behavioural Intention regarding LMS usage. However, performance mainly depends on an individual’s ability to use the system. When novel adopters anticipate the minimal effort required to use a system, they attach importance to it, positively influencing their acceptance intentions. The behavioural intentions of instructors and students to utilize LMS are also influenced by environmental and social intervention factors. Aside from the efficiency and usability of an information system, end users may only utilize it if they are motivated by influential others, who then influence their attitude and behaviour (Venkatesh, et al., 2003 ). The implication is that instructors and students rely more heavily on the encouragement of social acquaintances and relevant referents when deciding to use LMS for pedagogical purposes.

Self-Efficacy and Attitude as personality factors have been found to have a significant association with successful LMS use intentions in numerous studies (Al-Mamary, 2022 ). A productive self-efficacy and positive attitude towards LMS use have a substantial effect on acceptance and vice-versa. Self-Efficacy and Attitude of instructors and students are influenced by certain factors such as training and technical support.

As the most critical organizational factor, Technical Support and Training are the fourth most important factor. Technical Support and Training are regarded as crucial organizational factors that have been the subject of numerous studies (Alshammari, 2020 ), covers providing service to academicians and training them via workshops and practical courses to enhance their LMS (Zheng, et al., 2018 ). When academicians face a technical problem and receive no assistance from the IT unit, they will feel that working with an LMS is a waste of time because of the time it takes to resolve the problem (Baleghi-Zadeh, et al., 2017 ). As a result, if they don’t train well, it is possible that they will give up working with it altogether. The findings suggest that, in actuality, the likelihood of lecturers finding LMS usage simple will increase in proportion to the quality of the training and technical support offered to them by their institutions in order to assist them in the resolution of technical problems.

Quality factors such as System Quality, Service Quality and Information Quality come as the last vital factors listed in Fig.  2 . Quality factors are critical factors in predicting eLearning acceptance (Ababneh, 2016 ; Alharbi & Sandhu, 2018 ; Alshurideh, et al., 2021 ; Wang & Wang, 2009 ). It is essential for businesses involved in eLearning to improve the quality of learning management systems in order to boost the lecturers’ system usage, as well as to assist the lecturers in learning how to use their LMS with less time and effort. If the eLearning industries produce the LMS system with the highest possible quality, the LMS users may be able to limit the number of technical issues that occur. Additionally, there is a lack of training and connection between the IT unit and LMS users, which results in users being unaware of the significance of the system’s service quality.

The purpose of this study was to conduct a literature review of studies on learning management system acceptance and adoption, with the end goal of identifying the predominant models used by researchers to predict LMS acceptance in AGC. It provided a more in-depth explanation of the methodologies that were utilized for these studies. In the end, it investigated the findings of the milestones as well as the factors that influence LMS utilization in AGC. Like other studies, this study has limitations because it focused solely on LMS acceptance intentions in higher education in AGC and neglected to pay attention to schools and institutions. Other studies have addressed these limitations. In addition, the research concentrated solely on LMS as its primary technology of interest, ignoring all other technologies in the process. The findings of this study stress the importance of putting in place support systems that will make it much simpler to use LMS. Because the researchers were unable to locate any studies that had been carried out in the educational context of Bahrain, it could be suggested that future research should concentrate on factors that are specific to that country. The finding of this study suggests that future studies should investigate factors like governmental policy, pedagogical beliefs, and organizational culture. These are all factors that were not present in the studies that were reviewed, but the researchers believe that they are significant. In addition, the research suggests that future studies ought to place a greater emphasis on making use of mixed method design in order to discover acceptance and adoption factors in LMS research about AGC. In addition, upcoming research ought to make greater use of UTAUT on usage intentions of LMS in AGC and investigate moderators with contextual value. In addition, the intention of instructors to use learning management systems should be a primary focus of future research.

Summary of the reviewed studies

Data availability

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

• Ababneh H. Extending the technology acceptance model and critical success factors model to predict the use of cloud computing [Article] Journal of Information Technology Research. 2016; 9 (3):1–17. doi: 10.4018/JITR.2016070101. [ CrossRef ] [ Google Scholar ]
• Abdullah MS, Toycan M. Analysis of the factors for the successful e-learning services adoption from education providers’ and students’ perspectives: A case study of private universities in Northern Iraq. Eurasia Journal of Mathematics Science and Technology Education. 2017; 14 (3):1097–1109. [ Google Scholar ]
• Abdulmohson A, Kadhim MF, Anssari H, Al-Jobouri AA. Cost analysis of on-premise versus cloud-based implementation of Moodle in Kufa University during the pandemic. Indonesian Journal of Electrical Engineering and Computer Science. 2022; 25 (3):1787–1794. doi: 10.11591/ijeecs.v25.i3.pp1787-1794. [ CrossRef ] [ Google Scholar ]
• Ahmed, T. M., & Seliaman, M. E. (2017). Investigating the adoption and impact of e-learning in KSA: Prince Sattam bin Abdulaziz university case study. Journal of Theoretical and Applied Information Technology , 95 (11).
• Al Mulhem, A. (2020). Exploring the key factors in the Use of an E-Learning system among students at King Faisal University, Saudi Arabia. International Journal of Interactive Mobile Technologies .
• Al-Busaidi, K. A., & Al-Shihi, H. (2010). Instructors’ acceptance of learning management systems: A theoretical framework. Communications of the IBIMA , 2010 (2010), 1–10.
• Al-Hajri SA, Ghayas S, Echchabi A. Investigating the e-learning acceptance in Oman: Application of structural equation modelling approach. Journal of Computer Science. 2018; 14 (3):368–375. doi: 10.3844/jcssp.2018.368.375. [ CrossRef ] [ Google Scholar ]
• Al-Mamary YHS. Understanding the use of learning management systems by undergraduate university students using the UTAUT model: Credible evidence from Saudi Arabia. International Journal of Information Management Data Insights. 2022; 2 (2):100092. doi: 10.1016/j.jjimei.2022.100092. [ CrossRef ] [ Google Scholar ]
• Alammary, A., Alshaikh, M., & Alhogail, A. (2021). The impact of the COVID-19 pandemic on the adoption of e-learning among academics in Saudi Arabia. Behaviour & Information Technology , 1–23. 10.1080/0144929X.2021.1973106.
• Alfalah, A. A. (2023). Factors influencing students’ adoption and use of mobile learning management systems (m-LMSs): A quantitative study of Saudi Arabia [Article]. International Journal of Information Management Data Insights , 3 (1), 10.1016/j.jjimei.2022.100143. Article 100143.
• Alhabeeb A, Rowley J. E-learning critical success factors: Comparing perspectives from academic staff and students. Computers & Education. 2018; 127 :1–12. doi: 10.1016/j.compedu.2018.08.007. [ CrossRef ] [ Google Scholar ]
• Alharbi S, Drew S. Using the technology acceptance model in understanding academics’ behavioural intention to use learning management systems. International Journal of Advanced Computer Science and Applications. 2014; 5 (1):143–155. doi: 10.14569/IJACSA.2014.050120. [ CrossRef ] [ Google Scholar ]
• Alharbi H, Sandhu K. Explaining and predicting continuance usage intention of e-learning recommender systems: An empirical evidence from Saudi Arabia. International Journal of Business Information Systems. 2018; 29 (3):297–323. doi: 10.1504/ijbis.2018.095565. [ CrossRef ] [ Google Scholar ]
• Alharthi S, Levy Y, Awaji M. Empirical testing of resistance and misuse factors contributing to Instructors’ Use of E-Learning Systems in Saudi Arabia. AIS Transactions on Replication Research. 2019; 5 :1–14. doi: 10.17705/1atrr.00033. [ CrossRef ] [ Google Scholar ]
• Alhumsi MH, Alshaye RA. Applying technology acceptance model to Gauge University students’ perceptions of using blackboard in learning academic writing [Article] Knowledge Management and E-Learning. 2021; 13 (3):316–333. doi: 10.34105/j.kmel.2021.13.017. [ CrossRef ] [ Google Scholar ]
• Alkharang, M., & Ghinea, G. (2013). Factors influencing the adoption of e-learning in Kuwait. E-Learn: World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education
• Almaiah MA, Al-Khasawneh A, Althunibat A. Exploring the critical challenges and factors influencing the E-learning system usage during COVID-19 pandemic [Article] Education and Information Technologies. 2020; 25 (6):5261–5280. doi: 10.1007/s10639-020-10219-y. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Alshammari SH. The influence of technical support, perceived self-efficacy, and instructional design on students’ use of learning management systems [Article] Turkish Online Journal of Distance Education. 2020; 21 (3):112–141. doi: 10.17718/TOJDE.762034. [ CrossRef ] [ Google Scholar ]
• Alshammari SH, Ali MB, Rosli MS. The Influences of Technical Support, Self Efficacy and Instructional Design on the usage and Acceptance of LMS: A Comprehensive Review. Turkish Online Journal of Educational Technology-TOJET. 2016; 15 (2):116–125. [ Google Scholar ]
• Alshehri AH, Alahmari SA. Faculty e-Learning adoption during the COVID-19 pandemic: A case study of Shaqra University [Article] International Journal of Advanced Computer Science and Applications. 2021; 12 (10):855–862. doi: 10.14569/IJACSA.2021.0121095. [ CrossRef ] [ Google Scholar ]
• Alshurideh, M. T., Kurdi, A., AlHamad, B., Salloum, A. Q., Alkurdi, S. A., Dehghan, S., Abuhashesh, A., M., & Masa’deh, R. (2021). e. Factors Affecting the Use of Smart Mobile Examination Platforms by Universities’ Postgraduate Students during the COVID-19 Pandemic: An Empirical Study. Informatics , 8 (2), 32. https://www.mdpi.com/ 2227-9709/8/2/32
• Alsswey, A., & Al-Samarraie, H. (2019). M-learning adoption in the Arab gulf countries: A systematic review of factors and challenges. Education and Information Technologies, 24 (5), 3163–3176. 10.1007/s10639-019-09923-1.
• Alsswey A, Al-Samarraie H, El-Qirem FA, Zaqout F. M-learning technology in Arab Gulf countries: A systematic review of progress and recommendations. Education and Information Technologies. 2020 doi: 10.1007/s10639-019-10097-z. [ CrossRef ] [ Google Scholar ]
• Altalbe A. Antecedents of actual usage of e-Learning system in High Education during COVID-19 pandemic: Moderation effect of instructor support. Ieee Access: Practical Innovations, Open Solutions. 2021; 9 :93119–93136. doi: 10.1109/ACCESS.2021.3087344. [ CrossRef ] [ Google Scholar ]
• Alturise, F. (2022). Influence of teachers’ ICT skills on the adoption of an e-learning management system in Sport Psychology during the COVID-19 pandemic [Article]. Revista de Psicologia del Deporte , 31 (3), 87–100. https://www.scopus.com/inward/record.uri?eid=2-s2.0-85145895355&partnerID=40&md5=eefc83bfd 5d3c090ebb8def7374242f5
• Asiri MJS. Factors influencing the use of learning management system in saudi Arabian higher education: A theoretical framework. Higher Education Studies. 2012; 2 (2):125–137. doi: 10.5539/hes.v2n2p125. [ CrossRef ] [ Google Scholar ]
• Baleghi-Zadeh S, Ayub AFM, Mahmud R, Daud SM. The influence of system interactivity and technical support on learning management system utilization. Knowledge Management & E-Learning: An International Journal. 2017; 9 (1):50–68. [ Google Scholar ]
• Bervell B, Umar IN. A decade of LMS acceptance and adoption research in Sub-Sahara African higher education: A systematic review of models, methodologies, milestones and main challenges. Eurasia Journal of Mathematics Science and Technology Education. 2017; 13 (11):7269–7286. doi: 10.12973/ejmste/79444. [ CrossRef ] [ Google Scholar ]
• Binyamin, S. S., Rutter, M. J., & Smith, S. (2019). Extending the Technology Acceptance Model to understand students’ use of Learning Management Systems in Saudi Higher Education. International Journal of Emerging Technologies in Learning , 14 (3).
• Cao G, Shaya N, Enyinda CI, Abukhait R, Naboush E. Students’ relative attitudes and relative intentions to Use E-Learning Systems [Article] Journal of Information Technology Education: Research. 2022; 21 :115–136. doi: 10.28945/4928. [ CrossRef ] [ Google Scholar ]
• Chen W, Hirschheim R. A paradigmatic and methodological examination of information systems research from 1991 to 2001. Information systems journal. 2004; 14 (3):197–235. doi: 10.1111/j.1365-2575.2004.00173.x. [ CrossRef ] [ Google Scholar ]
• Cheng SL, Chen SB, Chang JC. Examining the multiplicative relationships between teachers’ competence, value and pedagogical beliefs about technology integration. British Journal of Educational Technology. 2021; 52 (2):734–750. doi: 10.1111/bjet.13052. [ CrossRef ] [ Google Scholar ]
• Creswell, J. W. (2012). Qualitative inquiry and research design: Choosing among five approaches (Vol. null).
• Creswell, J. W., & Creswell, D. (2018). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches (Fifth Edition ed.). SAGE.
• Daouk L, Aldalaien M. The usage of E-Learning Instructional Technologies in Higher Education Institutions in the United Arab Emirates (UAE) Turkish Online Journal of Educational Technology-TOJET. 2019; 18 (3):97–109. [ Google Scholar ]
• Davis FD, Bagozzi RP, Warshaw PR. User acceptance of computer technology: A comparison of two theoretical models. Management science. 1989; 35 (8):982–1003. doi: 10.1287/mnsc.35.8.982. [ CrossRef ] [ Google Scholar ]
• Dishaw MT, Strong DM. Extending the technology acceptance model with task–technology fit constructs. Information & Management. 1999; 36 (1):9–21. doi: 10.1016/S0378-7206(98)00101-3. [ CrossRef ] [ Google Scholar ]
• El-Masri M, Tarhini A. Factors affecting the adoption of e-learning systems in Qatar and USA: Extending the Unified Theory of Acceptance and Use of Technology 2 (UTAUT2) Educational Technology Research and Development. 2017; 65 (3):743–763. doi: 10.1007/s11423-016-9508-8. [ CrossRef ] [ Google Scholar ]
• Fattah SA, Mousa AH, Mohsen MK, Khalaf SD, Mousa SH. Determinants of e-learning adoption in higher education in Iraq an academics and students’ perspective [Article] Telkomnika (Telecommunication Computing Electronics and Control) 2022; 20 (1):201–211. doi: 10.12928/TELKOMNIKA.v20i1.21550. [ CrossRef ] [ Google Scholar ]
• Gamage SHPW, Ayres JR, Behrend MB. A systematic review on trends in using Moodle for teaching and learning. International Journal of STEM Education. 2022; 9 (1):9. doi: 10.1186/s40594-021-00323-x. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Gautreau C. Motivational factors affecting the integration of a learning management system by faculty. Journal of Educators Online. 2011; 8 (1):n1. doi: 10.9743/JEO.2011.1.2. [ CrossRef ] [ Google Scholar ]
• Granić A, Marangunić N. Technology acceptance model in educational context: A systematic literature review. British Journal of Educational Technology. 2019; 50 (5):2572–2593. doi: 10.1111/bjet.12864. [ CrossRef ] [ Google Scholar ]
• Hussein MH, Ow SH, Ibrahim I, Mahmoud MA. Measuring instructors continued intention to reuse Google Classroom in Iraq: A mixed-method study during COVID-19. Interactive Technology and Smart Education. 2021; 18 (3):380–402. doi: 10.1108/ITSE-06-2020-0095. [ CrossRef ] [ Google Scholar ]
• Jamil LS. Assessing the behavioural intention of students towards learning management system, through technology acceptance model-case of iraqi universities. Journal of Theoretical and Applied Information Technology. 2017; 95 (16):3825–3840. [ Google Scholar ]
• Kunene KE, Maphosa C. An analysis of factors affecting utilisation of Moodle Learning Management System by Open and Distance Learning students at the University of Eswatini. Humanities. 2020; 5 (1):17–32. [ Google Scholar ]
• Kurdi BA, Alshurideh M, Salloum SA, Obeidat ZM, Al-dweeri RM. An empirical investigation into examination of factors influencing university students’ behavior towards elearning acceptance using SEM approach [Article] International Journal of Interactive Mobile Technologies. 2020; 14 (2):19–41. doi: 10.3991/ijim.v14i02.11115. [ CrossRef ] [ Google Scholar ]
• Mailizar M, Burg D, Maulina S. Examining university students’ behavioural intention to use e-learning during the COVID-19 pandemic: An extended TAM model. Education and Information Technologies. 2021; 26 (6):7057–7077. doi: 10.1007/s10639-021-10557-5. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Mohamed Riyath MI, Muhammed Rijah UL. Adoption of a learning management system among educators of advanced technological institutes in Sri Lanka. Asian Association of Open Universities Journal. 2022; 17 (2):161–177. doi: 10.1108/AAOUJ-03-2022-0032. [ CrossRef ] [ Google Scholar ]
• Moher D, Liberati A, Tetzlaff J, Altman DG, The PG. Preferred reporting items for systematic reviews and Meta-analyses: The PRISMA Statement. PLOS Medicine. 2009; 6 (7):e1000097. doi: 10.1371/journal.pmed.1000097. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Mouakket S, Bettayeb AM. Investigating the factors influencing continuance usage intention of learning management systems by university instructors. International Journal of Web Information Systems. 2015; 11 (4):491–509. doi: 10.1108/IJWIS-03-2015-0008. [ CrossRef ] [ Google Scholar ]
• Mujalli, A., Khan, T., & Almgrashi, A. (2022). University Accounting Students and Faculty Members Using the Blackboard Platform during COVID-19; Proposed Modification of the UTAUT Model and an Empirical Study. Sustainability , 14 (4), 2360. https://www.mdpi.com /2071-1050/14/4/2360
• Park SY. An analysis of the technology acceptance model in understanding university students’ behavioral intention to use e-learning. Educational Technology & Society. 2009; 12 (3):150–162. [ Google Scholar ]
• Phan, T. T. T., Vu, C. T., Doan, P. T. T., Luong, D. H., Bui, T. P., Le, T. H., & Nguyen, D. H. (2022). Two decades of studies on learning management system in higher education: A bibliometric analysis with Scopus database 2000–2020. Journal of University Teaching & Learning Practice , 19 (3), 09. https://doi.org/NA
• Rabaa’i AA, AlMaati SA, Zhu X. Students’ continuance intention to use moodle: An expectation-confirmation model approach [Article] Interdisciplinary Journal of Information Knowledge and Management. 2021; 16 :397–434. doi: 10.28945/4842. [ CrossRef ] [ Google Scholar ]
• Saleem, N. E., Al-Saqri, M. N., & Ahmad, S. E. (2016). Acceptance of Moodle as a teaching/learning tool by the faculty of the department of information studies at Sultan Qaboos University, Oman based on UTAUT. International Journal of Knowledge Content Development & Technology , 6 (2), 5–27. 10.5865/IJKCT.2016.6.2.005
• Salloum SA, Al-Emran M, Shaalan K, Tarhini A. Factors affecting the E-learning acceptance: A case study from UAE. Education and Information Technologies. 2019; 24 (1):509–530. doi: 10.1007/s10639-018-9786-3. [ CrossRef ] [ Google Scholar ]
• Sarrab M, Al-Khanjari Z, Alnaeli S, Bourdoucen H. 2018//). Human factors considerations in Mobile Learning Management Systems. Cham: Interactive Mobile Communication Technologies and Learning; 2018. [ Google Scholar ]
• Shishakly R. A further understanding of the Dominant factors affecting E-learning usage resources by students in universities in the UAE [Article] Eurasia Journal of Mathematics Science and Technology Education. 2021; 17 (11):1–15. doi: 10.29333/ejmste/11234. [ CrossRef ] [ Google Scholar ]
• Sinclair J, Aho AM. Experts on super innovators: Understanding staff adoption of learning management systems. Higher Education Research & Development. 2018; 37 (1):158–172. doi: 10.1080/07294360.2017.1342609. [ CrossRef ] [ Google Scholar ]
• Sulaiman TT, Mahomed ASB, Abd Rahman A, Hussan M. Factors affecting University lecturers’ adoption of Learning Management System (LMS) in Kurdistan Region of Iraq: A conceptual Framework. International Journal of Psychosocial Rehabilitation. 2019; 23 (2):860–871. doi: 10.37200/IJPR/V23I2/PR190336. [ CrossRef ] [ Google Scholar ]
• Sulaiman TT, Mahomed B, Rahman AS, Hassan M. Examining the influence of the Pedagogical Beliefs on the Learning Management System usage among University lecturers in the Kurdistan Region of Iraq. Heliyon. 2022; 8 (6):e09687. doi: 10.1016/j.heliyon.2022.e09687. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Sulaiman, T. T., Mahomed, A. S. B., Rahman, A. A., & Hassan, M. (2023). Understanding Antecedents of Learning Management System Usage among University Lecturers Using an Integrated TAM-TOE Model [Article]. Sustainability (Switzerland) , 15 (3), Article 1885. 10.3390/su15031885
• Suzanne, S., Emma, C., & Tsakani Violet, N. (2019). Lecturers’ Perceptions of Learning Management Systems Within a Previously Disadvantaged University. In P. Patricia Ordóñez de, D. L. Miltiadis, Z. Xi, & C. Kwok Tai (Eds.), Opening Up Education for Inclusivity Across Digital Economies and Societies (pp. 1–28). IGI Global. 10.4018/978-1-5225-7473-6.ch001
• Tondeur J, van Braak J, Ertmer PA, Ottenbreit-Leftwich A. Understanding the relationship between teachers’ pedagogical beliefs and technology use in education: A systematic review of qualitative evidence [journal article] Educational Technology Research and Development. 2017; 65 (3):555–575. doi: 10.1007/s11423-016-9481-2. [ CrossRef ] [ Google Scholar ]
• Venkatesh V, Bala H. Technology Acceptance Model 3 and a Research Agenda on Interventions. Decision Sciences. 2008; 39 (2):273–315. doi: 10.1111/j.1540-5915.2008.00192.x. [ CrossRef ] [ Google Scholar ]
• Venkatesh V, Davis FD. A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management science. 2000; 46 (2):186–204. doi: 10.1287/mnsc.46.2.186.11926. [ CrossRef ] [ Google Scholar ]
• Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS quarterly , 425–478.
• Venkatesh, V., Thong, J. Y., & Xu, X. (2012). Consumer acceptance and use of information technology: Extending the unified theory of acceptance and use of technology. MIS quarterly , 157–178.
• Wang, Wang CC. An empirical study of instructor adoption of web-based learning systems. Computers & Education. 2009; 53 (3):761–774. doi: 10.1016/j.compedu.2009.02.021. [ CrossRef ] [ Google Scholar ]
• Weber, A. S., & Hamlaoui, S. (2018). E-Learning in the Middle East and North Africa (MENA) Region . Springer International Publishing. https://books.google.com.my/books?id=mWBODwAAQBAJ .
• Zheng Y, Wang J, Doll W, Deng X, Williams M. The impact of organisational support, technical support, and self-efficacy on faculty perceived benefits of using learning management system. Behaviour & Information Technology. 2018; 37 (4):311–319. doi: 10.1080/0144929X.2018.1436590. [ CrossRef ] [ Google Scholar ]
• Ziraba A, Akwene GC, Lwanga SC. The adoption and use of Moodle Learning Management System in Higher Institutions of Learning: A systematic literature review. American Journal of Online and Distance Learning. 2020; 2 (1):1–21. [ Google Scholar ]
• Zwain AAA. Technological innovativeness and information quality as neoteric predictors of users’ acceptance of learning management system: An expansion of UTAUT2. Interactive Technology and Smart Education. 2019; 16 :239–254. doi: 10.1108/ITSE-09-2018-0065. [ CrossRef ] [ Google Scholar ]
• Zwain, A. A. A., & Haboobi, M. N. H. (2019). Investigating determinants of Faculty and Students’ Acceptance of E-Learning Management Systems using UTAUT2. International Journal of Innovation Creativity and Change , 7 (8).

Adoption of a learning management system among educators of advanced technological institutes in Sri Lanka

Asian Association of Open Universities Journal

ISSN : 2414-6994

Article publication date: 12 August 2022

Issue publication date: 5 October 2022

The study investigates the factors that impact the adoption of learning management systems (LMSs) among educators for effective implementation of open and distance learning (ODL) environment in advanced technological institutes (ATIs).

Design/methodology/approach

This study uses the extended technology acceptance model (TAM) and analyses data using the partial least square–based structural equation modelling approach to validate the construct and test proposed hypotheses. Data were collected through an online questionnaire from the respondents.

This study reveals that perceived self-efficacy and job relevance significantly impact perceived usefulness (PU) and perceived ease of use (PEU). PU, PEU and service quality significantly impact attitudes of educators, which impact their behavioural intention and actual use of LMS as a chain reaction.

Practical implications

The management should organise hands-on training sessions to improve educators' computer self-efficacy and explain the importance of the LMS and its features to offer an effective ODL environment for delivering high-quality education.

Originality/value

The previous studies focused on LMS use from the students' point of view rather than educators. This study investigates educators' LMS adoption in ATIs using the extended TAM. The findings may be helpful for management to implement an effective ODL environment that offers fully integrated distance learning and e-learning during the prevailing COVID-19 pandemic.

• System quality

Mohamed Riyath, M.I. and Muhammed Rijah, U.L. (2022), "Adoption of a learning management system among educators of advanced technological institutes in Sri Lanka", Asian Association of Open Universities Journal , Vol. 17 No. 2, pp. 161-177. https://doi.org/10.1108/AAOUJ-03-2022-0032

Emerald Publishing Limited

Copyright © 2022, Mohamed Ismail Mohamed Riyath and Uthuma Lebbe Muhammed Rijah

Published in the Asian Association of Open Universities Journal . Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) license. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this license may be seen at http://creativecommons.org/licences/by/4.0/ legalcode

Introduction

Every industry has been influenced by the widespread use of information and communication technology (ICT) which has brought various advancements in the education industry. These technological advancements offer educators and students greater prospects of customising teaching and learning ( Ratheeswari, 2018 ). The rapid growth of internet availability and ICT encourages educational institutes to integrate e-learning applications to ensure the continuous delivery of academic programs and student interaction ( Ashrafzadeh and Sayadian, 2015 ). Teaching and learning can be made more interactive and effective with the help of e-learning technology. One popular technology supporting e-learning is the learning management system (LMS) ( Coskuncay, 2013 ).

The LMS is a web-based application that integrates and organises all teaching and learning initiatives. LMS use significantly lowers the costs and complexity of knowledge transfer within an organisation ( Pelet, 2013 ). Many higher-educational institutes (HEIs) currently use the LMS as an essential element for their course delivery ( Browne et al. , 2006 ; Alhazmi and Rahman, 2012 ; Washington, 2019 ), and it has become an indispensable tool in higher education for the interactive teaching and learning process ( Pelet, 2013 ; Alturki and Aldraiweesh, 2021 ). The rising popularity of e-learning, distance education and blended learning and the increased use of the LMS pressure HEIs to deliver high-quality courses online ( Alomari et al. , 2020 ). The LMS largely supports traditional face-to-face teaching and is considered the backbone of e-learning at HEIs ( Washington, 2019 ). Educators and students in HEIs are often mandated to adopt the LMS ( Shine and Heath, 2020 ). Educators use the LMS to streamline their students' learning activities. It facilitates educators to share course materials, communicate with students and assess their performance. Educators must engage and interact with students using a suitable LMS to offer a better learning environment ( Yen et al. , 2018 ). Many HEIs implement the LMS to enhance the quality of teaching and learning; hence, they provide training on technical skills to users and motivate them to be more interactive ( Rhode et al. , 2017 ).

Teaching and learning become more interactive due to the effective use of the LMS ( Waheed et al. , 2016 ; Alshammari, 2020 ; Alshammari et al. , 2016 ). During the COVID-19 pandemic, teaching and learning were physically interrupted in most HEIs and educators were compelled to switch to open and distance learning (ODL) modes. This study focuses on advanced technological institutes (ATIs) functioning under the Sri Lanka Institute of Advanced Technological Education (SLIATE). This institute is a leading HEI in the country, working under the portfolio of the Ministry of Education of Sri Lanka ( Gunasekara, 2015 ). SLIATE has implemented Moodle LMS to support face-to-face teaching and learning for the last decade ( Dona et al. , 2013 ). ATIs foster advanced technical education at the post-secondary level in each district of the island. They offer Higher National Diploma (HND) programs in various academic disciplines, including engineering, agriculture, information technology, business and languages. The administration and academic affairs of the ATIs are coordinated by a centralised system managed by SLIATE. Therefore, course curriculum design and implementation and semester-end examinations are undertaken by the SLIATE, while teaching and learning are conducted on the campus (ATIs) of SLIATE under a common academic calendar.

However, LMS use is unsatisfactory among educators and students in ATIs ( Jayathilake and Jayawardhana, 2017 ). Perera (2019) states that only 50% of educators use the LMS in ATIs. ATIs switched to the ODL mode during the pandemic to continue academic activities using the LMS and virtual conferencing applications (VCAs). Nevertheless, ATIs have been struggling to effectively implement the ODL mode due to educators' underuse of the LMS. Therefore, the top management of the ATIs needs to motivate the educators to use the LMS to implement ODL to effectively offer a quality teaching and learning environment. The top management of ATIs should understand critical factors influencing LMS adoption among educators to motivate them to use the LMS. Therefore, this study investigates the factors affecting LMS adoption among educators in ATIs to implement ODL effectively. This study employs a well-known theoretical framework: technological acceptance model (TAM). Many researchers consider the TAM as an appropriate model for investigating the factors affecting user intention and use of technology ( Mailizar et al. , 2021 ; Jayathilake and Jayawardhana, 2017 ; Abdullah et al. , 2016 ).

This paper consists of six sections. The first section is the introduction that gives an overview of the key roles of the LMS and the purpose of this study. The second section is a literature review that focuses on recent literature on LMS, TAM, and the development of hypotheses. Data collection, research design, and conceptual framework are described in the third section. Data analysis and discussions are presented in the fourth and fifth sections. Finally, the last section details the conclusion and implication.

Literature review

Learning management system.

The LMS is one of the most widely used web-based applications, and its use in HEIs is burgeoning ( Dutta et al. , 2013 ). The LMS includes several integrated technologies for delivering and administering ODL. There are two types of LMSs available: open source (e.g. Moodle, Forma LMS, Open edX, etc.) and commercial (e.g. Google Classroom, Blackboard, Docebo LMS, etc.). Most LMSs are adaptable, simple to use, accessible and user-friendly ( Alturki and Aldraiweesh, 2016 ; Arsovic and Stefanovic, 2020 ). Educators can use the LMS to develop online course content and then monitor it to improve critical reasoning skills and encourage students to work together on activities in university ( Zanjani et al. , 2016 ). The LMS comprises many features, including video conferencing, online group chats, live comments, lecture resources and the interaction between the teacher and the student. Learning modules, course evaluations and grading are available in the LMS, and all of them may be customised to meet teaching and learning needs ( Walker et al. , 2016 ). Non-traditional modes of teaching and learning assisted by online approaches to education have a favourable impact on both educators and students ( Anshari et al. , 2017 ). Educators use the LMS to share course content and teaching materials with students, as well as to promote collaboration and participation among students via the use of virtual forums. Students are encouraged to engage, share opinions, discuss issues and comment on ideas presented by their colleagues ( Goh et al. , 2014 ). The Moodle is an open-source, free LMS online platform extensively used by several HEIs to engage students and develop more comprehensive and interactive course materials ( Dhika et al. , 2020 ; Devi and Aparna, 2020 ; Nagi et al. , 2008 ). Moodle LMS is widely used in almost all HEIs in Sri Lanka ( Tennakoon and Lasanthika, 2021 ; Hasmy, 2020 ).

Technology acceptance model (TAM)

Davis initially proposed the TAM in 1989, which investigates the elements that have been identified as effects on human behaviour in adopting information systems (ISs). According to the TAM, the actual use (AU) of ISs is affected by the user's behavioural intention (BI), which is affected by the user's attitude (ATT). The ATT is affected by perceived usefulness (PU) and perceived ease of use (PEU). PEU influences PU. Researchers apply the TAM in numerous situations by adding new constructs. These extensions may be categorised into three areas: adding components from related models, adding more belief structures and evaluating predictors of PU and PEU ( Wixom and Todd, 2005 ). Davis (1989) defines PU as “the degree to which a person believes that using a particular system would enhance his/her job performance”. He defines PEU as “the degree to which a person believes that using a particular system would be free of physical and mental effort”.

Buabeng-Andoh and Baah (2020) investigated the intention of pre-service teachers to use the LMS using the TAM. The finding of the study is that ATT and social influence (SI) significantly affect BI to use the LMS. However, facilitating conditions (FCs) do not affect BI to use the LMS. Goh et al. (2014) conducted another study to determine academics' intentions to use the LMS. They find that PU positively supports their intention to use the LMS, but PEU does not. According to Holzmann et al. (2020) , teachers' use of technology depends on various factors. As per the study, the FC, PEU and ATT significantly affect teachers' technology use intention, although SI and effort expectancy (EE) do not. Likewise, Onaolapo and Oyewole (2018) investigated the effect of PE, EE and FC on learners' technology use in education. Their study's findings reveal that PE, EE and FC are strongly associated with learners' technology use. Fathema et al. (2015) examined the LMS use of educators in HEIs using the extended TAM and show that educators' attitudes about LMSs are significantly influenced by the three suggested external variables: perceived self-efficacy (PSE), system quality (SQ) and facilitating conditions (FC).

According to Holden and Rada (2011) , teachers' technology SE influences their use of technology. Panda and Mishra (2007) report that faculty members believed that poor internet connectivity and insufficient training are the key challenges to e-learning adoption, followed by organisational rules and instructional design. They find that faculty adoption of e-learning was mostly driven by a personal interest in using technology, intellectual challenge and adequate technical infrastructure. According to Mokhtar et al. (2018) , the BI of instructors to use the LMS is directly affected by task–technology fit (TTF), PU and PEU. Meanwhile, TTF, compatibility, convenience, SE, personal innovativeness (PI) and subjective norm (SN) significantly influence PU and PEU. Many previous studies have investigated LMS use from the view of students ( Saroia and Gao, 2019 ; Ashrafi et al. , 2020 ). However, limited research has looked into this topic from the view of educators ( Mokhtar et al. , 2018 ). Since educators' LMS use is vital to students' engagement in the learning process through course content creation and sharing, learners' LMS use behaviours can be influenced. As a result, it is imperative to investigate educators' intentions to use the LMS.

Furthermore, many studies have been conducted regarding using e-learning and LMS adoption among students and educators employing various adoption models ( Wrycza and Kuciapski, 2018 ; Uğur and Turan, 2018 ; Bervell and Umar, 2017 ; Sharma et al. , 2017 ). These studies failed to accommodate the variables job relevance (JR) ( Siyam, 2019 ; Saroia and Gao, 2019 ; Hong et al. , 2021 ), PSE ( Park et al. , 2012 ; Thongsri et al. , 2020 ; Abdullah et al. , 2016 ) and SQ ( Rughoobur-Seetah and Hosanoo, 2021 ; Mailizar et al. , 2021 ; Abdullah et al. , 2016 ) in their adoption models, even though these variables affect the adoption of ISs. Furthermore, few studies have been conducted in Sri Lanka from educators' point of view ( Gunasinghe et al. , 2020 ), and there are no studies in the existing literature in the context of non-degree-awarding, state-owned HEIs. This study fills the literature gap by using the TAM with its six original and the three new variables to investigate the factors affecting the educators' adoption of the LMS in ATIs.

PEU significantly impacts PU of the LMS.

PEU significantly impacts ATT towards using the LMS.

PU significantly impacts ATT towards using the LMS.

PU significantly impacts BI to use the LMS.

ATT significantly impacts BI to use the LMS.

BI significantly impacts the AU of the LMS.

The constructs PEU and PU may not be enough, and additional factors may be required in the TAM to comprehensively investigate IS adoption ( Siyam, 2019 ). Therefore, three external factors were identified after evaluating the relevant literature: JR ( Venkatesh and Davis, 2000 ), SE ( Tam and Cheung, 2020 ; Park et al. , 2012 ; Thongsri et al. , 2020 ; Chen and Tseng, 2012 ; Abdullah et al. , 2016 ) and SQ ( Taat and Francis, 2020 ). The three proposed external constructs/variables with relevant literature to consider in the adopted conceptual framework of this study are given in detail with justifications in the following subsections.

Job relevance (JR)

JR significantly impacts the PU of the LMS.

JR significantly impacts the PEU of the LMS.

Perceived self-efficacy (PSE)

PSE significantly impacts the PU of the LMS.

PSE significantly impacts the PEU of the LMS.

System quality (SQ)

SQ significantly affects PU of the LMS.

SQ significantly affects users' ATT of the LMS.

Methodology

This study investigated educators of ATIs who volunteered to take part in this online survey. All the participants in this study were directly involved in teaching regular academic programs offered by the ATIs, which fit the study's purpose and context. The questionnaire was taken from Alharbi and Drew (2014) and adapted to fit the local research environment of the study. The questionnaire's face validity and content validity were ensured in the adaption phase by thoroughly assessing the relevant literature and incorporating comments and suggestions of a panel of experts in the field. This questionnaire comprises questions on demographic profiles in the first section and questions on educators' perceptions about the LMS in the second section. These questions were categorised into eight subsections based on the extended TAM-adapted conceptual framework: SQ, PSE, JR, PU, PEU, ATT, BI and AU. Respondents were required to respond to each question on a five-point Likert scale based on their degree of agreement (1: strongly disagree, 5: strongly agree). Due to the COVID-19 pandemic, the questionnaire was converted to Google Forms® and distributed through appropriate WhatsApp® groups, and responses were collected online. The questionnaire was active online for two weeks from 2 September 2021. The survey received responses from 197 educators of ATIs island wide; however, only 164 responses were usable for this study.

Data were analysed using partial least square–based structural equation modelling (PLS-SEM) to examine the conceptual frameworks' model validity and proposed hypotheses. PLS-SEM is a more appropriate approach to examining complex models with many latent constructs and smaller samples ( Akter et al. , 2017 ; Hair et al. , 2017 ; Hair et al. , 2019 ). Therefore, the proposed model for this study was analysed using SmartPLS® 3.2. The two-step approach of Schumacker et al. (2015) was used to analyse the data in the model: measurement model and structural equation model. The measurement model assessed the observed items' reliability and validity with associated latent constructs. In the measurement model, the construct reliability, convergent validity (CV) and discriminant validity (DV) were evaluated. The structural equation model was used to test the proposed hypothesis in the adapted conceptual framework of this study ( Figure 1 ). The bootstrap strategy with 5,000 subsamples was used to determine the significance of the path coefficients of the structural equation model.

Data analysis and findings

Descriptive statistics.

The survey respondents' demographic profile is shown in Table 1 . The total number of valid respondents was 164, of which females were 60.4%, while males were 39.6%. Most respondents (43.3%) were above 45 years old, 39% were between 30 and 45 years old and 17.7% were less than 30 years old. Of all the respondents, 37.8% have over 15 years of teaching experience, 36.0% had between 5 and 15 years of experience and 26.2% had less than 5 years of experience. Furthermore, regarding experience in using the LMS, 49.4% had less than 2 years, 17.1% had 2–5 years, 10.4% had over 5 years and 23.2% had no experience. Finally, 81.7% of the respondents were academic staff, whereas 18.3% were academic support staff.

Measurement models

Cronbach's alpha (CA) and composite reliability (CR) were estimated to determine the construct reliability of each construct. Hair et al. (2019) recommend that CA and CR values should be greater than 0.70 to consider a construct as a reliable one. Table 2 indicates that the values of all the constructs exceeded the threshold value, suggesting that all constructs are reliable and have internal consistency. The constructs' CV was assessed using the average variance extracted (AVE) value. Table 2 shows the results; the AVE value exceeds the threshold value of 0.5, as Hair et al. (2019) suggested, and the CV of all constructs was confirmed. It confirms the validity of the internal structure of the construct. The DV was measured using cross-loadings and Fornell and Larcker (1981) criteria. Table 3 indicates that the cross-loadings for each item of respective constructs are more than 0.5. It confirms the inner construct validity with accepted parameters, as proposed by Hair et al. (2019) . Table 4 indicates that all the diagonal values are higher than those in the remaining values in respective columns, confirming DV for all constructs, as Fornell and Larcker (1981) suggested. Therefore, it confirms that all constructs in the model satisfy the reliability and validity thresholds and are suitable for further analysis.

Structural model and hypothesis testing

The structural model was tested using PLS-SEM analysis. This study used bootstrapping procedure in SmartPLS with 5,000 subsamples to generate respective t-statistics and p -values of regression path coefficient to test the proposed hypotheses. Figure 2 depicts the estimated structured equation model in the bootstrap procedure, and Table 5 shows the hypothesis test results at a 95% confidence interval. This analysis indicates that all the proposed hypotheses are true except H1 since the t -values are greater than 1.96 and p -values are below 0.05.

The path coefficients show that PSE significantly impacts PU ( β  = 0.299, t  = 3.737, p  < 0.000) and PEU ( β  = 0.552, t  = 9.691, p  < 0.000), confirming H9 and H10 ; JR significantly impacts PU ( β  = 0.135, t  = 2.048, p  < 0.041) and PEU ( β  = 0.238, t  = 3.746, p  < 0.000), confirming H7 and H8 ; SQ significantly impacts PU ( β  = 0.311, t  = 4.262, p  < 0.000) and ATT ( β  = 0.208, t  = 2.706, p  < 0.007), confirming H11 and H12 ; PU significantly impacts ATT ( β  = 0.296, t  = 3.930, p  < 0.000) and BI ( β  = 0.378, t  = 5.166, p  < 0.000), confirming H3 and H4 ; PEU significantly impacts ATT ( β  = 0.297, t  = 3.999, p  < 0.000), confirming H2 , and ATT significantly impacts BI ( β  = 0.347, t  = 4.363, p  < 0.000), confirming H5 . BI significantly impacts AU ( β  = 0.653, t  = 15.526, p  < 0.000), confirming H6 ; PEU does not significantly impact PU ( β  = 0.153, t  = 1.719, p  < 0.086), and therefore, H1 was rejected.

The coefficient of determination ( R 2 ) measures how much the independent variables explain the variances in the dependent variable in a linear regression model ( Chicco et al. , 2021 ; Rodríguez Sánchez et al. , 2019 ). Table 6 shows the structural equation model's coefficient of determination ( R 2 ). It indicates that the model explains a significant amount of the variance in all of the dependent variables: PU (51.7%), PEU (45.4%), ATT (46.7%), BI (41.9%) and AU (42.6%). Falk and Miller (1992) suggest that R 2 should be greater than 0.10. Therefore, all dependent variables meet Falk and Miller (1992) criteria in this model. However, as there is a significant portion of unexplained variations in the model, additional crucial factors might be included to improve the prediction strength of the model.

Many exogenous variables can impact the endogenous variable in a conceptual framework. The removal of an exogenous variable might impact the endogenous variable. F -square is the change in R -square that occurs when an exogenous variable is omitted from the framework ( Aberson, 2019 ). Table 7 shows the result of the F -square. According to Cohen (1988) , F -square is the effect size (0.02 is small, 0.15 is medium and 0.35 is large) ( Yıldırım and Güler, 2021 ). Accordingly, this study reveals that the ATT → BI ( F 2  = 0.134), JR → PEU ( F 2  = 0.091), JR → PU ( F 2  = 0.029), PEU → ATT ( F 2  = 0.098), PEU → PU ( F 2  = 0.024), PSE → PU ( F 2  = 0.105), PU → ATT ( F 2  = 0.091), SQ → ATT ( F 2  = 0.045) and SQ → PU ( F 2  = 0.122) have small effects; PU → BI ( F 2  = 0.159) and PU → BI ( F 2  = 0.159) have medium effects and BI → AU ( F 2  = 0.742), PSE → PEU ( F 2  = 0.491) have large effects.

The predictive power determines the predictive strength of endogenous constructs. Predictive power is evaluated in this study by executing Stone-Geisser's Q 2 ( Geisser, 1974 ). Hair et al. (2019) suggested that Q 2 values greater than zero suggest that the model is effectively rebuilt and has predictive power. Table 8 shows the Q 2 for each endogenous construct in this model, showing good predictive power.

The study finds that SQ positively impacts PU, consistent with earlier research ( Maheshwari, 2021 ; Abdullah et al. , 2016 ; Fathema and Sutton, 2013 ). This finding puts forward that SQ aspects like ease of access, system availability to fulfil user needs and system flexibility of the LMS are essential and contribute to the PU of the LMS. This impliedly tells that educators in ATIs seriously pay attention to LMS quality. Therefore, the SQ of the existing LMS should be improved to enhance the engagement of educators in ATIs. Furthermore, Saarinen (1996) quoted, “High system quality requires a good user interface and, in the long run, flexibility, allowing changes in the processing style, and adaptation to new requirements.” It proposes that if the system matches user requirements, it has enough functionality to accomplish the goals of adopting the LMS by educators for an effective e-learning system in ATIs. In addition, this study finds that SQ positively impacts ATT of educators towards the LMS. It advocates that if the implemented LMS meets all requirements of educators with greater flexibility, their attitude towards the LMS will be improved.

Furthermore, this research reveals that PSE positively impacts PU and PEU, stating that PSE is identified to influence the belief of educators and behaviour towards the LMS significantly. Users with a favourable perspective of computer SE assures that the system is simple and can quickly fix issues. In addition, this finding emphasises that if educators are competent in using computers and other digital devices, they perceive LMS as user-friendly and a more robust tool for delivering course contents. This result is consistent with that of earlier research ( Chen and Tseng, 2012 ; Abdullah et al. , 2016 ; Thongsri et al. , 2020 ; Alammary et al. , 2014 ). Moreover, JR positively impacts PEU and PU, stating that educators who believe the LMS is an effective and relevant tool for fulfilling their job will find it helpful and user-friendly. This finding is consistent with that of earlier research, suggesting that if the technology is relevant to their job and assists them in fulfilling their duties, they will consider it a supporting tool that raises their PU and PEU ( Saroia and Gao, 2019 ).

In addition, this research identifies that SQ, PU and PEU significantly impact educators' ATT of an LMS. It argues that strong positive beliefs of educators in quality aspects, usefulness and accessibility of an LMS make favourable attitudes towards using the LMS in teaching. Additionally, PU is the stronger predictor of ATT than SQ and PSU, implying that the degree of belief of educators in the usefulness of the LMS largely impacts their attitude towards using the LMS. This finding is consistent with that of earlier research that confirms the relationships ( Mailizar et al. , 2021 ; Mou et al. , 2017 ; Hamid et al. , 2016 ).

According to the TAM literature, PEU impacts PU ( Ong, 2019 ; Mukminin et al. , 2020 ). However, the data analysis of this study reveals that the impact is not statistically significant. It evidences that the belief of educators about the usefulness of the LMS is not influenced by its ease of accessibility. The possible reason for this is that all educators in ATIs are well educated and familiar with operating any applications. Hence, operating the LMS is not a complex phenomenon for them. Therefore, this study highlights that ease of access does not mean the LMS is a handy tool for effective pedagogy. Therefore, the management should build the capacity of their staff about the notable advanced features of the LMS for effective teaching.

The proposed model of this study explains that 42.6% of the variations imply that the model accurately predicts LMS use by educators. The PU is the dominant predictor of BI, followed by ATT. This indicates that if educators find the LMS more suitable, more comfortable to use, more helpful and simpler for teaching, their BI to use the LMS will be high. This is consistent with earlier research that confirms the relationships ( Fearnley and Amora, 2020 ; Mailizar et al. , 2021 ). Educators' BI directly affects the AU of the LMS, and this finding is in line with those of earlier research ( Fearnley and Amora, 2020 ; Munabi et al. , 2020 ; Fathema et al. , 2015 ). This indicates that educators with positive attitudes towards the LMS have a higher level of BI, which results in a higher level of actual use of the LMS in ATIs.

Conclusion and implication

Teacher–student interaction is vital to offering quality education. The LMS is an excellent tool to interact with students and engage them in learning activities. LMS use among ATIs' educators is unsatisfactory, and educators have poor interaction with students even after the implementation of ODL due to the pandemic. This study intends to identify the factors influencing LMS adoption among ATIs' educators to offer an effective ODL environment. This study has proposed a conceptual framework based on the TAM with three new external variables – PSE, JR and SQ – to achieve the objective of the study. The findings assert that the framework used in this study performs well in explaining the factors influencing the adoption of the LMS among educators of ATIs in Sri Lanka. PSE and JR significantly impact PU and PEU of the LMS. In addition to PU and PEU, SQ significantly impacts the ATT of educators towards the LMS. PU and ATT significantly impact educators' BI and AU of the LMS. However, PEU has no significant impact on PU. The finding of this study confirms the previous empirical studies that use the TAM.

The findings will highly be helpful to ATIs' top management as they prepare to adopt an effective ODL environment that offers fully integrated distance learning and e-learning during and after the COVID-19 pandemic. These findings have significant practical implications: First, ATIs' management should encourage and facilitate educators to implement ODL through effective use of the LMS. ATIs implemented Moodle LMS, a web application hosted in the cloud. Web applications are frequently updated with new features and come to the market in a short period. It requires an up-to-date high level of internet skills, which affects LMS adoption. Hence, the management should organise hands-on training sessions to improve computer SE and internet skills of educators and explain the importance of the LMS with the latest versions and features. Furthermore, ongoing technical guidance should be arranged to handle various user issues. Second, the system designers should concentrate on the contents and functionalities when designing and developing the LMS. The designers should study deep user requirements to effectively design the LMS, including the display size and system suitability, system integration, interactive media support, learner control, diversity of communication and test types, and user responsiveness. The designers should evaluate the quality and availability of information to enhance the experience of educators while responding to and promoting the benefits of using the LMS.

This study has some limitations. First, the proposed model explains nearly half of the total variations. It suggests that the next half of the total variance is unexplained. Second, this study collected only 164 responses, a relatively small sample. Third, the hypothesised relationship among the construct could be moderated by other variables like gender, age, prior experience, academic discipline, etc. These moderating variables are not considered when assessing the model in this study. Therefore, we propose that future studies use a model incorporating additional meaningful constructs affecting LMS adoption and moderating variables with a reasonable sample size and thereby, the new model could explain more variances in LMS adoption.

Conceptual framework

Estimated path coefficients

Demographic profile

Convergent validity indicators

Discriminant validity

Factor and cross-loadings

Hypothesis test results

Coefficient of determination ( R 2 )

Effect size ( f 2 )

Predictive power ( Q 2 )

Note(s): SSE – sum of squared error; SSO – sum of squares of observations

Abdullah , F. , Ward , R. and Ahmed , E. ( 2016 ), “ Investigating the influence of the most commonly used external variables of TAM on students' Perceived Ease of Use (PEOU) and Perceived Usefulness (PU) of e-portfolios ”, Computers in Human Behavior , Vol.  63 C , pp.  75 - 90 .

Aberson , C.L. ( 2019 ), Applied Power Analysis for the Behavioral Sciences , 2nd ed., Routledge , New York .

Akter , S. , Fosso Wamba , S. and Dewan , S. ( 2017 ), “ Why PLS-SEM is suitable for complex modelling? An empirical illustration in big data analytics quality ”, Production Planning and Control , Vol.  28 Nos 11-12 , pp.  1011 - 1021 .

Alamin , A.A. , Wilkin , C.L. , Yeoh , W. and Warren , M. ( 2020 ), “ The impact of self-efficacy on accountants' behavioral intention to adopt and use accounting information systems ”, Journal of Information Systems , Vol.  34 No.  3 , pp.  31 - 46 .

Alammary , A. , Sheard , J. and Carbone , A. ( 2014 ), “ Blended learning in higher education: three different design approaches ”, Australasian Journal of Educational Technology , Vol.  30 No.  4 , pp.  440 - 454 .

Alharbi , S. and Drew , S. ( 2014 ), “ Using the technology acceptance model in understanding academics' behavioural intention to use learning management systems ”, International Journal of Advanced Computer Science and Applications , Vol.  5 No.  1 , pp.  143 - 155 .

Alhazmi , A.K. and Rahman , A.A. ( 2012 ), “ Why LMS failed to support student learning in higher education institutions ”, 2012 IEEE Symposium on e-learning, e-management and e-services , Kuala Lumpur, Malaysia , pp.  1 - 5 .

Alomari , M.M. , El-Kanj , H. , Alshdaifat , N.I. and Topal , A. ( 2020 ), “ A framework for the impact of human factors on the effectiveness of learning management systems ”, IEEE ACCESS , Vol.  8 , pp.  23542 - 23558 .

Alshammari , S.H. ( 2020 ), “ The influence of technical support, perceived self-efficacy, and instructional design on students' use of learning management systems ”, Turkish Online Journal of Distance Education , Vol.  21 No.  3 , pp.  112 - 141 .

Alshammari , S.H. , Ali , M.B. and Rosli , M.S. ( 2016 ), “ The influences of technical support, self-efficacy and instructional design on the usage and acceptance of LMS: a comprehensive review ”, Turkish Online Journal of Educational Technology-TOJET , Vol.  15 No.  2 , pp.  116 - 125 .

Alturki , U.T. and Aldraiweesh , A. ( 2016 ), “ Evaluating the usability and accessibility of LMS “blackboard” at King Saud University ”, Contemporary Issues in Education Research (CIER) , Vol.  9 No.  1 , pp.  33 - 44 .

Alturki , U. and Aldraiweesh , A. ( 2021 ), “ Application of learning management system (LMS) during the COVID-19 pandemic: a sustainable acceptance model of the expansion technology approach ”, Sustainability , Vol.  13 No.  19 , pp.  1 - 16 .

Anshari , M. , bin Alas , Y. and Guan , L.S. ( 2017 ), “ Pervasive knowledge, social networks, and cloud computing: e-learning 2.0 ”, Eurasia Journal of Mathematics, Science and Technology Education , Vol.  11 No.  5 , pp.  909 - 921 .

Arsovic , B. and Stefanovic , N. ( 2020 ), “ E-learning based on the adaptive learning model: case study in Serbia ”, Sadhana-academy Proceedings in Engineering Sciences , Vol.  45 No.  1 , pp.  1 - 22 .

Ashrafi , A. , Zareravasan , A. , Rabiee Savoji , S. and Amani , M. ( 2020 ), “ Exploring factors influencing students’ continuance intention to use the learning management system (LMS): a multi-perspective framework ”, Interactive Learning Environments , pp. 1 - 23 , doi: 10.1080/10494820.2020.1734028 .

Ashrafzadeh , A. and Sayadian , S. ( 2015 ), “ University instructors' concerns and perceptions of technology integration ”, Computers in Human Behavior , Vol.  49 August , pp.  62 - 73 , 2015 .

Bakhsh , M. , Mahmood , A. and Sangi , N.A. ( 2017 ), “ Examination of factors influencing students and faculty behavior towards m-learning acceptance ”, The International Journal of Information and Learning Technology , Vol.  34 No.  3 , pp.  166 - 188 .

Bervell , B. and Umar , I.N. ( 2017 ), “ Validation of the UTAUT model: Re-considering non-linear relationships of exogeneous variables in higher education technology acceptance research ”, Eurasia Journal of Mathematics, Science and Technology Education , Vol.  13 No.  10 , pp.  6471 - 6490 .

Browne , T. , Jenkins , M. and Walker , R. ( 2006 ), “ A longitudinal perspective regarding the use of VLEs by higher education institutions in the United Kingdom ”, Interactive Learning Environments , Vol.  14 No.  2 , pp.  177 - 192 .

Buabeng-Andoh , C. and Baah , C. ( 2020 ), “ Pre-service teachers' intention to use learning management system: an integration of UTAUT and TAM ”, Interactive Technology and Smart Education , Vol.  17 No.  4 , pp.  455 - 474 .

Chen , H.-R. and Tseng , H.-F. ( 2012 ), “ Factors that influence acceptance of web-based e-learning systems for the in-service education of junior high school teachers in Taiwan ”, Evaluation and Program Planning , Vol.  35 No.  3 , pp.  398 - 406 .

Chicco , D. , Warrens , M.J. and Jurman , G. ( 2021 ), “ The coefficient of determination R -squared is more informative than SMAPE, MAE, MAPE, MSE and RMSE in regression analysis evaluation ”, PeerJ Computer Science , Vol.  7 , p. e623 .

Cohen , J. ( 1988 ), Statistical Power Analysis for the Behavioral Sciences , Routledge .

Coskuncay , F. ( 2013 ), “ A model for instructors' adoption of learning management systems: empirical validation in higher education context ”, Turkish Online Journal of Educational Technology , Vol.  12 No.  2 , pp.  13 - 25 .

Davis , F.D. ( 1989 ), “ Perceived usefulness, perceived ease of use, and user acceptance of information technology ”, MIS Quarterly , Vol.  13 No.  3 , pp.  319 - 340 .

Devi , K.S. and Aparna , M. ( 2020 ), “ Moodle–An effective learning management system for 21st century learners ”, Alochana Chakra Journal , Vol.  9 No.  6 , pp.  4474 - 4485 .

Dhika , H. , Destiawati , F. , Sonny , M. and Jaya , M. ( 2020 ), “ Comparison of learning management system Moodle, Edmodo and Jejak Bali ”, International Conference on Progressive Education (ICOPE 2019), Bandar Lampung, Indonesia , Vol.  422 , pp.  90 - 94 .

Dona , K.L. , Keppell , M. and Warusawitharana , A. ( 2013 ), “ Gazing into the future of Sri Lankan higher education: capacity building for the future ”, ASCILITE-Australian Society for Computers in Learning in Tertiary Education Annual Conference .

Dutta , A. , Roy , R. and Seetharaman , P. ( 2013 ), “ Course management system adoption and usage: a process theoretic perspective ”, Computers in Human Behavior , Vol.  29 No.  6 , pp.  2535 - 2545 .

Falk , R.F. and Miller , N.B. ( 1992 ), A Primer for Soft Modeling , University of Akron Press .

Fathema , N. and Sutton , K.L. ( 2013 ), “ Factors influencing faculty members' Learning Management Systems adoption behavior: an analysis using the Technology Acceptance Model ”, International Journal of Trends in Economics Management and Technology (IJTEMT) , Vol.  2 No.  6 , pp.  20 - 28 .

Fathema , N. , Shannon , D. and Ross , M. ( 2015 ), “ Expanding the technology acceptance model (TAM) to examine faculty use of learning management systems (LMSs) in higher education institutions ”, Journal of Online Learning and Teaching , Vol.  11 No.  2 , pp.  210 - 232 .

Fearnley , M.R. and Amora , J.T. ( 2020 ), “ Learning management system adoption in higher education using the extended technology acceptance model ”, IAFOR Journal of Education , Vol.  8 No.  2 , pp.  89 - 106 .

Fornell , C. and Larcker , D.F. ( 1981 ), “ Evaluating structural equation models with unobservable variables and measurement error ”, Journal of Marketing Research , Vol.  18 No.  1 , pp.  39 - 50 .

Geisser , S. ( 1974 ), “ A predictive approach to the random effect model ”, Biometrika , Vol.  61 No.  1 , pp.  101 - 107 .

Goh , W.W. , Hong , J.L. and Gunawan , W. ( 2014 ), “ Exploring lecturers' perceptions of learning management system: an empirical study based on TAM ”, International Journal of Engineering Pedagogy , Vol.  4 No.  3 , pp.  48 - 54 .

Gunasekara , N. ( 2015 ), “ Learning management system for SLIATE ”, Information Technology, University of Colombo School of Computing , available at: https://dl.ucsc.cmb.ac.lk/jspui/handle/123456789/3213 .

Gunasinghe , A. , Hamid , J.A. , Khatibi , A. and Azam , S.M.F. ( 2020 ), “ The adequacy of UTAUT-3 in interpreting academician's adoption to e-Learning in higher education environments ”, Interactive Technology and Smart Education , Vol.  17 No.  1 , pp.  86 - 106 .

Hair , J.F. , Jr , Matthews , L.M. , Matthews , R.L. and Sarstedt , M. ( 2017 ), “ PLS-SEM or CB-SEM: updated guidelines on which method to use ”, International Journal of Multivariate Data Analysis , Vol.  1 No.  2 , pp.  107 - 123 .

Hair , J.F. , Risher , J.J. , Sarstedt , M. and Ringle , C.M. ( 2019 ), “ When to use and how to report the results of PLS-SEM ”, European Business Review , Vol.  31 No.  1 , pp.  2 - 24 .

Hamid , A.A. , Razak , F.Z.A. , Bakar , A.A. and Abdullah , W.S.W. ( 2016 ), “ The effects of perceived usefulness and perceived ease of use on continuance intention to use e-government ”, Procedia Economics and Finance , Vol.  35 No.  2016 , pp.  644 - 649 .

Hasmy , A.J.M. ( 2020 ), “ Effective use of collaboration tools in Moodle LMS by lecturers and students at South Eastern University of Sri Lanka ”, Journal of Information Systems and Information Technology (JISIT) , Vol.  5 No.  1 , pp.  101 - 109 .

Holden , H. and Rada , R. ( 2011 ), “ Understanding the influence of perceived usability and technology self-efficacy on teachers' technology acceptance ”, Journal of Research on Technology in Education , Vol.  43 No.  4 , pp.  343 - 367 .

Holzmann , P. , Schwarz , E.J. and Audretsch , D.B. ( 2020 ), “ Understanding the determinants of novel technology adoption among teachers: the case of 3D printing ”, The Journal of Technology Transfer , Vol.  45 No.  1 , pp.  259 - 275 .

Hong , X. , Zhang , M. and Liu , Q. ( 2021 ), “ Preschool teachers' technology acceptance during the COVID-19: an adapted technology acceptance model ”, Frontiers in Psychology , Vol.  12 , p. 2113 .

Jayathilake , M. and Jayawardhana , R. ( 2017 ), “ Applying the technology acceptance model (TAM) to understand adaptation of e-learning in alternative higher education sector in Sir Lanka, case study: the Sri Lanka institute of advanced technological education (SLIATE) ”, International Conference on the Humanities (ICH), 2017 Faculty of Humanities , University of Kelaniya, Sri Lanka , p. 51 .

Maheshwari , G. ( 2021 ), “ Factors affecting students' intentions to undertake online learning: an empirical study in Vietnam ”, Education and Information Technologies , Vol.  26 , pp.  6629 - 6649 .

Mailizar , M. , Burg , D. and Maulina , S. ( 2021 ), “ Examining university students' behavioural intention to use e-learning during the COVID-19 pandemic: an extended TAM model ”, Education and Information Technologies , Vol.  26 , pp.  7057 - 7077 .

Mokhtar , S.A. , Katan , H. and Hidayat-ur-Rehman , I. ( 2018 ), “ Instructors' behavioural intention to use learning management system: an integrated TAM perspective ”, TEM Journal , Vol.  7 No.  3 , p. 513 .

Mou , J. , Shin , D.-H. and Cohen , J. ( 2017 ), “ Understanding trust and perceived usefulness in the consumer acceptance of an e-service: a longitudinal investigation ”, Behaviour and Information Technology , Vol.  36 No.  2 , pp.  125 - 139 .

Mukminin , A. , Habibi , A. , Muhaimin , M. and Prasojo , L.D. ( 2020 ), “ Exploring the drivers predicting behavioral intention to use m-learning management system: partial least square structural equation model ”, IEEE Access , Vol.  8 , pp.  181356 - 181365 .

Munabi , S.K. , Aguti , J. and Nabushawo , H.M. ( 2020 ), “ Using the TAM model to predict undergraduate distance learners behavioural intention to use the Makerere University learning management system ”, Open Access Library Journal , Vol.  7 No.  9 , pp.  1 - 12 .

Nagi , K. , Suesawaluk , P. and U-Lan , P.V. ( 2008 ), “ Evaluating interactivity of eLearning resources in A learning management system (LMS)-A case study of MOODLE, an open source platform ”, The Fifth International Conference on eLearning for Knowledge-Based Society , Bangkok, Thailand , pp.  772 - 776 .

Onaolapo , S. and Oyewole , O. ( 2018 ), “ Performance expectancy, effort expectancy, and facilitating conditions as factors influencing smart phones use for mobile learning by postgraduate students of the University of Ibadan, Nigeria ”, Interdisciplinary Journal of E-Skills and Lifelong Learning , Vol.  14 No.  1 , pp.  95 - 115 .

Ong , C.Y.F.S. ( 2019 ), “ Malaysian undergraduates' behavioural intention to use LMS: an extended self-directed learning technology acceptance model (SDLTAM) ”, Journal of ELT Research , Vol.  4 No.  1 , pp.  8 - 25 .

Panda , S. and Mishra , S. ( 2007 ), “ E-learning in a mega open University: faculty attitude, barriers and motivators ”, Educational Media International , Vol.  44 No.  4 , pp.  323 - 338 .

Park , S.Y. , Nam , M.W. and Cha , S.B. ( 2012 ), “ University students' behavioral intention to use mobile learning: evaluating the technology acceptance model ”, British Journal of Educational Technology , Vol.  43 No.  4 , pp.  592 - 605 .

Pelet , J.E. ( 2013 ), E-learning 2.0 Technologies and Web Applications in Higher Education , Information Science Reference .

Perera , K. ( 2019 ), “ Approaches for establishing a sustainable quality assurance system for Sri Lanka Institute of Advanced Technological Education ”, Accelerating Higher Education Expansion and Development (AHEAD) .

Ratheeswari , K. ( 2018 ), “ Information communication technology in education ”, Journal of Applied and Advanced Research , Vol.  3 No.  1 , pp.  45 - 47 .

Rhode , J. , Richter , S. , Gowen , P. , Miller , T. and Wills , C. ( 2017 ), “ Understanding faculty use of the learning management system ”, Online Learning , Vol.  21 No.  3 , pp.  68 - 86 .

Rodríguez Sánchez , A. , Salmerón Gómez , R. and García , C. ( 2019 ), “ The coefficient of determination in the ridge regression ”, Communications in Statistics - Simulation and Computation , pp. 1 - 19 , doi: 10.1080/03610918.2019.1649421 .

Rughoobur-Seetah , S. and Hosanoo , Z.A. ( 2021 ), “ An evaluation of the impact of confinement on the quality of e-learning in higher education institutions ”, Quality Assurance in Education , Vol.  29 No.  4 , pp.  422 - 444 .

Saarinen , T. ( 1996 ), “ An expanded instrument for evaluating information system success ”, Information and Management , Vol.  31 No.  2 , pp.  103 - 118 .

Saroia , A.I. and Gao , S. ( 2019 ), “ Investigating university students' intention to use mobile learning management systems in Sweden ”, Innovations in Education and Teaching International , Vol.  56 No.  5 , pp.  569 - 580 .

Schumacker , R.E. , Lomax , R.G. and Schumacker , R. ( 2015 ), A Beginner's Guide to Structural Equation Modeling , Taylor & Francis .

Sharma , S.K. , Gaur , A. , Saddikuti , V. and Rastogi , A. ( 2017 ), “ Structural equation model (SEM)-Neural Network (NN) model for predicting quality determinants of e-learning management systems ”, Behaviour and Information Technology , Vol.  36 No.  10 , pp.  1053 - 1066 .

Shine , B. and Heath , S.E. ( 2020 ), “ Techniques for fostering self-regulated learning via learning management systems in on-campus and online courses ”, Journal of Teaching and Learning with Technology , Vol.  9 No.  1 , pp.  119 - 126 .

Singh , A. , Sharma , S. and Paliwal , M. ( 2020 ), “ Adoption intention and effectiveness of digital collaboration platforms for online learning: the Indian students' perspective ”, Interactive Technology and Smart Education , Vol.  18 No.  4 , pp.  493 - 514 .

Siyam , N. ( 2019 ), “ Factors impacting special education teachers' acceptance and actual use of technology ”, Education and Information Technologies , Vol.  24 No.  3 , pp.  2035 - 2057 .

Taat , M.S. and Francis , A. ( 2020 ), “ Factors influencing the students' acceptance of e-learning at teacher education institute: an exploratory study in Malaysia ”, International Journal of Higher Education , Vol.  9 No.  1 , pp.  133 - 141 .

Tam , K.M. and Cheung , S.Y. ( 2020 ), “ Measuring physical activity self-efficacy, self-regulation, social support among Hong Kong working adults: a validation study ”, Journal of Physical Education , Vol.  7 No.  1 , pp.  66 - 73 .

Tennakoon , W. and Lasanthika , W. ( 2021 ), “ Evaluating E-learning systems success: a case of Sri Lanka ”, Wayamba Journal of Management , Vol.  12 No.  1 , pp.  59 - 96 .

Thongsri , N. , Shen , L. and Bao , Y. ( 2020 ), “ Investigating academic major differences in perception of computer self-efficacy and intention toward e-learning adoption in China ”, Innovations in Education and Teaching International , Vol.  57 No.  5 , pp.  577 - 589 .

Uğur , N.G. and Turan , A.H. ( 2018 ), “ Retracted article: E-learning adoption of academicians: a proposal for an extended model ”, Behaviour and Information Technology , Vol.  37 No.  4 , pp.  393 - 405 .

Venkatesh , V. and Davis , F.D. ( 2000 ), “ A theoretical extension of the technology acceptance model: four longitudinal field studies ”, Management Science , Vol.  46 No.  2 , pp.  186 - 204 .

Waheed , M. , Kaur , K. , Ain , N. and Hussain , N. ( 2016 ), “ Perceived learning outcomes from Moodle: an empirical study of intrinsic and extrinsic motivating factors ”, Information Development , Vol.  32 No.  4 , pp.  1001 - 1013 .

Walker , D.S. , Lindner , J.R. , Murphrey , T.P. and Dooley , K. ( 2016 ), “ Learning management system usage ”, Quarterly Review of Distance Education , Vol.  17 No.  2 , pp.  41 - 50 .

Washington , G.Y. ( 2019 ), “ The learning management system matters in face-to-face higher education courses ”, Journal of Educational Technology Systems , Vol.  48 No.  2 , pp.  255 - 275 .

Wixom , B.H. and Todd , P.A. ( 2005 ), “ A theoretical integration of user satisfaction and technology acceptance ”, Information Systems Research , Vol.  16 No.  1 , pp.  85 - 102 .

Wrycza , S. and Kuciapski , M. ( 2018 ), “ Determinants of academic E-learning systems acceptance ”, in Cham , pp.  68 - 85 .

Yen , S.-C. , Lo , Y. , Lee , A. and Enriquez , J. ( 2018 ), “ Learning online, offline, and in-between: comparing student academic outcomes and course satisfaction in face-to-face, online, and blended teaching modalities ”, Education and Information Technologies , Vol.  23 No.  5 , pp.  2141 - 2153 .

Yıldırım , M. and Güler , A. ( 2021 ), “ Positivity explains how COVID-19 perceived risk increases death distress and reduces happiness ”, Personality and Individual Differences , Vol.  168 No.  2021 , p.  110347 .

Zanjani , N. , Edwards , S.L. , Nykvist , S. and Geva , S. ( 2016 ), “ LMS acceptance: the instructor role ”, The Asia-Pacific Education Researcher , Vol.  25 No.  4 , pp.  519 - 526 .

Acknowledgements

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Corresponding author

Related articles, we’re listening — tell us what you think, something didn’t work….

Report bugs here

All feedback is valuable

Please share your general feedback

Join us on our journey

Platform update page.

Visit emeraldpublishing.com/platformupdate to discover the latest news and updates

Questions & More Information

Answers to the most commonly asked questions here

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

Learning Management Systems in the Workplace- A Literature Review.pdf

2018, 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE)

Learning Management Systems (LMSs) are a vital software platform to deliver education and training courses online. They enable the creation, management and delivery of educational content making it easier for businesses of all sizes and types to administer educational content. Like any system, LMS also needs to be user-friendly and easily usable. Usability is a measure of the degree to which users can use a product or system to effectively, efficiently, and satisfactorily achieve their objectives; this means that users will be trained quickly and efficiently if the degree of usability of LMS is high. This paper attempts to discover the essential usability factors, implementation and adoption issues and the barriers and enablers within the LMS domain, primarily in workplace settings. To achieve these aims, a review of the literature has been carried out by considering 23 research articles published between 2014 to 2018. The discussion highlighted current issues in the field, as well as gaps and possibilities for further research.

Related Papers

2010 43rd Hawaii International Conference on System Sciences

Mehmet Onacan

Christos Katsanos , Nikolaos Tselios

THEMES in …

Petros Georgiakakis

Journal of global strategic management

Norkhairani Abdul Rawi

E-learning has become part of the learning approaches in all Higher Learning Institutions (HLI) in Malaysia. As outlined in the Globalized Online Learning (GOL), which is one of the ideas from 10 big ideas of education in Malaysia, 30 per cent of the delivery method should be online. Sustaining students’ commitment to actively engage in e-learning is a big challenge. Each university has their own e-learning platform that is used as means to deliver the teaching and learning materials. As for Universiti Sultan Zainal Abidin (UniSZA), KELIP is the Learning Management System (LMS) that has been customized to serve as one-stop center for lecturers and students. Even though it has been established for quite a long time, the percentage of usage especially among students is still low. Hence there is a need to identify why this situation occurs. This study is conducted to investigate the usability of KELIP from student perspective. The students were taught how to use the platform and partic...

Selvarajah Thuseethan

As far as Learning Management System is concerned, it offers an integrated platform for educational materials, distribution and management of learning as well as accessibility by a range of users including teachers, learners and content makers especially for distance learning. Usability evaluation is considered as one approach to assess the efficiency of e-Learning systems. It is used to evaluate how well technology and tools are working for users. There are some factors contributing as major reasons why the LMS is not used effectively and regularly. Learning Management Systems, as major part of e-Learning systems, can benefit from usability research to evaluate the LMS ease of use and satisfaction among its handlers. Many academic institutions worldwide prefer using their own customized Learning Management Systems; this is the case with Moodle, an open source LMS platform designed and operated by most of the universities in Sri Lanka. This paper gives an overview of Learning Management Systems used in Sri Lankan universities, and evaluates its usability using some pre-defined usability standards. In addition it measures the effectiveness of LMS by testing the Learning Management Systems. The findings and result of this study as well as the testing are discussed and presented.

Nouran Radwan

Many Universities recognize the necessity of using Learning Management Systems (LMSs) to increase learners' motivation, encourage interaction, provide feedback and provide support during the learning process. There are many proprietary and open source LMSs that can be found as alternative products. With the ever-growing number of LMSs, the task of developing and evaluating LMS becomes even more important. This paper discusses the factors that affect the use of LMS, the developing issues that have an impact on LMS and the evaluation processes that should be taken to select the suitable LMS. Also, it presents challenges that face LMS success, efficiency, assessment, evaluation, selection and usability. Last it shows the current trends to answer a question of how the LMS of the future will look like.

Problems and Perspectives in Management

Desmond Govender

With increasing demand for integrating learning management systems (LMS) into teaching and learning, a well-designed LMS is crucial. User interface evaluation has become a critical quality of interactive LMS intended to meet the requirements of users. This article investigates the effect of the interaction on the user and assesses the extent of system functionality. It further seeks to evaluate the interface’s success within the framework of fundamental human computer interface principles under a constructivist learning approach. Using an LMS assumes that when learners are engaged in a social learning context they actively construct knowledge; therefore, the resource is considered a tool to support learning and not an end in itself. The research investigates use of the LMS by two sets of users: staff members (module creators) and learners (module consumers), using semi-structured questionnaires and interviews. The research indicates that use of an LMS supports collaborative and auth...

IOER International Multidisciplinary Research Journal

IOER International Multidisciplinary Research Journal ( IIMRJ) , Louie Agustin

The Learning Management System (LMS) is a type of web-based software that is hosted on a server and is used to handle students' information, program enrollment, course content, and evaluation tools. The primary goal of this study was to ascertain the efficacy of learning management systems in affecting change in this new normal school context. The research study engaged 38 students from various courses who were enrolled in the Citi Global College's Weekend Education Program. A descriptive research design was adopted in this study. Descriptive research is one in which the range of one or more variables is described without respect for any causal or other assumption. The frequency (f) and percentage (%) relevant factors were used to determine the respondents' demographic profile in terms of age, sex, course, year level, available internet connection, and gadget use at home; whereas the weighted Mean (WM), ttest, and f-test variables were used to analyze the gathered data regarding the level of effectiveness on the quality characteristics of the learning management system and to determine if there were significant differences in respondents' assessments. The study's findings indicate that respondents were all working students; the majority of them use mobile data as their available internet subscription; and that there was no significant difference in the level of effectiveness of LMS's quality characteristics when respondents were grouped according to their demographic profile. The study recommended that instructors allow students sufficient time to process their output at the LMS because they were working students; prepare video tutorials to familiarize students with the LMS platform; use a good or faster internet connection of at least 4 Mbps and at least Android Marshmallow or iOS 8 on their mobile devices; and that the Learning Management System may be proposed for use by other neighboring schools.

Proceedings of the 19th International Conference on Enterprise Information Systems

Tayana Uchoa Conte

RELATED PAPERS

robert gjerdingen

Peterson, Andrew; Stahl, Garth; Soong, Hannah (Eds.)The Palgrave Handbook of Citizenship and Education

Sara Franch

francisco vargas

Daniel Galadza

Christina Mayerl

International Journal of Environment and Climate Change

yuvraj Gavali

Alifiulahtin Utaminingsih

Essays on Biblical Historiography: From Jeroboam II to John Hyrcanus I

Israel Finkelstein

Commonwealth Secretariat Local Government Reform

Roger Koranteng

Apresentação

Tatiana Carpanezzi

Diagnostic Microbiology and Infectious Disease

Mariana Kruger

Projeto História. Revista do Programa de Estudos …

Victor Lacerda

Interfaces Científicas - Saúde e Ambiente

Marianna Vidal

Encyclopedia of Lakes and Reservoirs

Andrew Gronewold

Archives of the Turkish Society of Cardiology

Journal of Engineering and Applied Sciences

Abdul Muhith

Louise Egan

Journal of Alzheimer's disease : JAD

Javier Sáez-Valero

一模一样加拿大范莎学院毕业证 uofs文凭证书英文录取通知原版一模一样

pkl T A N G E R A N G selatan

Armando Lopez (Miguel Armando López Ramírez)

Liliana Bezrodnik

Journal of The American Society of Nephrology

Kenneth Abreo

Drug Design, Development and Therapy

Jose Arbelo Hidalgo

RELATED TOPICS

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

Accessibility Links

• Skip to content
• Skip to search IOPscience
• Skip to Journals list
• Accessibility help
• Accessibility Help

Click here to close this panel.

Purpose-led Publishing is a coalition of three not-for-profit publishers in the field of physical sciences: AIP Publishing, the American Physical Society and IOP Publishing.

Together, as publishers that will always put purpose above profit, we have defined a set of industry standards that underpin high-quality, ethical scholarly communications.

We are proudly declaring that science is our only shareholder.

A Systematic Review for Online Learning Management System

Ng Syuan Xin 1 , Abdul Samad Shibghatullah 1 , Kasthuri A/P Subaramaniam 1 and Mohd Helmy Abd Wahab 2

Published under licence by IOP Publishing Ltd Journal of Physics: Conference Series , Volume 1874 , The 1st International Recent Trends in Engineering, Advanced Computing and Technology Conference (RETREAT) 2020 1-3 December 2020, Paris, France Citation Ng Syuan Xin et al 2021 J. Phys.: Conf. Ser. 1874 012030 DOI 10.1088/1742-6596/1874/1/012030

Article metrics

2176 Total downloads

Share this article

Author e-mails.

[email protected]

Author affiliations

1 Institute of Computer Science & Digital Innovation, UCSI University, Kuala Lumpur, Malaysia

2 Department of Computer Engineering, Faculty of Electrical and Electronic Engineering, Universiti Tun Hussein Onn Malaysia (UTHM), Malaysia

Buy this article in print

Currently, because of the Covid-19 pandemic there a lot of demand to use online Learning Management System (LMS). The LMS available usually offer similar features and is hard for user to choose which is appropriate for them. The key criteria for analysing LMS are learning skills tools, productivity tools and communication tools. The purpose of this paper is to do systematic review on the current LMS, the problem with current LMS and the potential solutions that might help. To serve this purpose five learning management system are chosen which are Moodle, Sakai, SumTotal, Blackboard and ATutor among the other learning management systems in the market. The reasons are it consists of open source platforms such as Moodle, ATutor and Sakai and commercial platforms such as Blackboard and SumTotal. The findings from this review is quite interesting and it can be used to help users such as university/colleges, and students in selecting their LMS. It also give some ideas to LMS developer in enhancing their LMS or proposing a new features.

Export citation and abstract BibTeX RIS

Content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence . Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.

• No results found

Learning management systems

2. literature review, 2.3 elearning, 2.3.4 learning management systems.

Currently, there is widespread use of LMSs like Moodle, WebCT and Blackboard (Chikh & Berkani, 2010; Vrazalic et al., 2009), which are important elements of elearning globally. According to Rogers et al. (2005), the term LMS applies to any use of web technology to

plan, organise, execute, and control the various aspects of the learning process. The same authors also quote Bersin’s (2005) definition (in Rogers et al., 2005) of an enterprise LMS, which is a single application utilised all over the enterprise to manage corporate training programmes. A LMS can also be called a course management system, a training

administration system, a training management system, or an integrated learning system (McArdle, Monahan, & Bertolotto, 2008; Rogers et al., 2005). But perhaps the most inclusive definition of LMS is that given by Greenberg (2002, cited by Rogers et al., 2005), which notes that an LMS is a high-level, deliberate solution for planning, implementing and managing all learning activities within an organisation, including virtual classrooms, online courses and instructor-guided courses. LMSs offer a platform for organisations’ online learning

environments by facilitating the management and implementation, as well as the

monitoring, of blended learning for a wide array of people including students, employees, customers and other stakeholders (Rogers et al., 2005).

Learning management systems such as Moodle, Blackboard and Sakai are procedural, whilst adaptive learning environments focus on learning itself (De Bra et al., 2013). De Bra et al. (2013) noted that over the last 15 years, the two complementary concepts have provided learning pathways and environments in education. Watson and Watson (2007) explained that the term learning management systems was derived from concepts characterised by an army of acronyms at the end of the twentieth century: CBI (computer-based instruction), CAI (computer-aided instruction), CAL (computer-assisted learning) and these variously described practice tasks, tutorial matter and perhaps individualised assistance through diversions from the main programme.

Jostens Learning, an American commercial provider, used the term integrated learning system to offer programmes beyond the curriculum delivery. These programmes included management instruction and further individual assistance and they were integrated with other systems (Bailey, 1993). Bailey described another product, learning management systems, which referred to the non-curricular material from the main product. However, the term was used for a variety of pedagogical matters. These included assessment and data;

communications between students and instructors; and student administration. A learning management system extends beyond these functions as follows:

 Instructional objectives are tied to individual lessons.  Lessons are incorporated into the standardized curriculum.

 A management system collects the results of student performance.  Lessons are provided based on the individual student’s learning progress

(Bailey, 1993, p.29).

This list includes the integration of pedagogy and the organisation’s administrative systems, such as finance and human resources (Bailey, 1993). Coates, James, and Baldwin (2005) noted that organisation-wide and internet-based learning management systems such as WebCT and Blackboard were having a profound effect on tertiary education, but LMSs have received little attention from researchers, other than research into the selection of

technology. Coates et al. (2005) noted that by 2005 the scope of online systems had extended beyond that envisaged in the early 1990s and incorporated:

 asynchronous and synchronous communication, incorporating e-mail announcements, list servers, chats and discussion

 curricular development and delivery including various resources and learning objectives

 assessment, both formative and summative  student and class administration.

Coates et al. (2005) explained that the management systems available at the time of their research were usually the products of universities, rather than the commercially developed. In Australia, Coates et al. (2005) pointed to the dominance of WebCT and Blackboard, which were then integrated into the majority of universities. The attraction of these systems was based on cost, access and reliability, although James et al. discussed andragogical

Such systems reduce cost overheads and lecture-room space, and facilitate access to student records and attainment, and thus make it easier to provide individual attention to students, and to incorporate assessment and quality standards within the programmes. Importantly, according to Coates et al. (2005), learning management systems enrich the student’s learning experience and remove levels of stress through such devices as continuous assessment and feedback.

A decade ago there were issues emerging from the learning management systems

experiences of universities. These were generally the same as those raised in the literature reviews cited above. Coates et al. (2005) noted that whilst assessments such as multiple- choice questions were useful to a point, they did not involve, as Schmid et al. (2014) advocated, engagement in meaningful activities. At the time, Coates et al. (2005) were also concerned about the evolution of student engagement, and whether the systems were seen as part of the university’s systems, or as an opportunity for engagement with the available resources. As this was a period when mobile devices and large-scale social media were becoming popular, the universities’ adoption of such massive systems may have appeared unremarkable to the ‘wired’ generation (Hekkert, Suurs, Negro, Kuhlmann, & Smits, 2007). The Blackboard LMS is one of the most common elearning technologies used by learning institutions across the world. Like other LMSs, Blackboard’s main functions include course/ content management, virtual classes, a discussion board, and other collaboration tools such as blogs, email, podcasts and wiki (Badawood & Steenkamp, 2012). In Saudi Arabia,

Blackboard is the most common LMS (Zouhair, 2010) and is, therefore, the LMS that this thesis will investigate. Its wide adoption is bolstered by its availability and early market penetration (Zouhair, 2010).

While LMSs have been adopted widely in higher education across the globe, it is evident that they involve the adoption of new technology, which can be a complex process

technology adoption, and these factors may also act as barriers to the adoption of LMSs as indicated in the following section.

2.3.4.1 Barriers to LMS

A review of the literature around LMSs revealed that the barriers to LMS adoption, although broad, can be grouped into four main types of issues: institutional, technological, academic and student-related issues. This categorisation draws upon Al-Busaidi and Al-Shihi’s (2010) finding that “the major issues that might influence instructors' acceptance of LMS might be related to the instructors' characteristics …, organisation factors … and the technology” (p. 4). In addition, student issues, such as lack of knowledge about information communication technology, have also been identified as one of the barriers to the use of learning

management systems in learning institutions (Nasser, Cherif, & Romanowski, 2011). Although at times there is overlap between the four types of issues, each of them is described separately below.

In relation to institutional issues, the barriers to using LMSs include poor access, or lack of access, to the technology and lack of incentives or programmes offered by the learning institutions to support the use of LMSs (Asiri et al., 2012; Fathema, Shannon, & Ross, 2015; Mtebe, 2015). For instance, in an analysis of the factors that influence the use of

instructional technology, Asiri et al. (2012) identified various issues that inhibit the use of technology for the purpose of delivering instruction in academic institutions. The

institutional factors that were identified by Asiri et al. (2012) include lack of staff development initiatives with respect to the use of technology, lack of policy and

administrative support, as well as lack of professional programmes to support or encourage the use of technology. Other institutional factors that have been identified as barriers to the use of LMSs include: high cost of implementation (Maina & Nzuki, 2015; Venter, van

Rensburg, & Davis, 2012); poor institutional decisions (Maina & Nzuki, 2015); lack of institutional policies on and instructional designs for elearning (Fathema et al., 2015); inadequate technical support (Azlim, Husain, Hussin, & Maksom, 2014; Maina & Nzuki,

2015; Fathema et al., 2015); and institutional technology training (Azlim et al., 2014; Fathema et al., 2015).

Technological issues that act as hindrances to the use of LMSs include software and hardware related problems, technical malfunctions of systems, internet access and availability problems, and network problems. In the context of Saudi Arabia’s higher

education Asiri et al. (2012) identified technological barriers as issues. They include internet access and availability as well as availability and accessibility of resources in the Arabic language. Similarly, it has been noted that a “lack of availability and accessibility of

technology” is one of the factors that makes it difficult to use technology for instruction and learning in learning institutions (Becker, Newton, & Sawang, 2013, p. 217). Another issue that acts as a major barrier to using LMSs is poor technological infrastructure, or a lack of it (Venter et al., 2012). This can particularly be said of developing countries like KSA, which generally have low levels of technological infrastructure. For instance, in Kenya, another developing country, a study by Tarus, Gichoya, and Muumbo (2015) found that inadequate elearning and information communication technology infrastructure is one of the key challenges that hinders the implementation of elearning in public universities. It has been argued that if KSA wants to have world-class universities, the country will need to invest heavily in technology and infrastructure (Colbran & Al-Ghreimil, 2013).

Academic issues that hinder the use of LMSs include lack of knowledge and experience in using technology, difficulties associated with the system, and lecturers’ attitudes towards the use of technology. These issues are also referred to as personal barriers and include “attitude toward technology, computer and internet experience, and technological skills and know-how” (Asiri et al. 2012, p. 128). Also, experience can determine intention and usage behaviour towards technology (Venkatesh, Morris, Davis, & Davis, 2003). A review of literature conducted by Fathema et al. (2015) found that the personal barriers to the use of elearning technologies include lack of knowledge and skills to use technology, lack of training, lack of role models, and the perception that elearning technologies are time- consuming. In addition, attitudinal barriers to the use of elearning technologies include lack

of faith in the technology, concerns about student access, and unwillingness to work with the technology (Fathema et al., 2015).

Lastly, student-related issues refers to matters such as student acceptance of the technology, accessibility of the technology to students, and students’ perceptions and attitudes towards the usefulness and perceived ease of use of the technology (Barczyk, Hixon, Buckenmeyer and Zamojski, 2012; Fathema et al., 2015; Logan & Neumann, 2010). For instance, Barczyk et al. (2012) note that students who encounter technology barriers (e.g. the inability to access or surf the internet) are less likely to easily adopt LMSs.

Specifically, Blackboard’s multi-layer folder system for the management of course materials has been criticised as being constraining to instructors and confusing to students (Logan & Neumann, 2010). Various reviews of literature (Fathema et al., 2015, p. 212) have also found “usefulness and ease of use to be good determinants of the student acceptance” of elearning technologies.

2.3.4.2 Enablers of LMSs

There are several reasons why institutions of higher learning have overcome most hindrances indicated above and adopted LMSs. These enablers (or motivators) have

enhanced the adoption of LMSs. The factors that promote the use of LMSs can be classified into four categories: improving teaching, improving student learning, improving working conditions of lecturers, and other reasons (TAM external and internal factors).

With regard to improving teaching, it can be said that faculty members are likely to use LMSs in their teaching activities if they know that the technology helps improve the teaching experience, both in terms of both helping the lecturers in their teaching activities and

improving students’ learning experiences. For example, in a study on how Moodle, another LMS, improves teaching, Thindwa (2016) found that “Moodle improves teaching quality as it has a high student satisfaction level, an aspect that is equated with teaching quality” (p. 64). This implies that members of faculty are more likely to use a LMS if it is associated with a high level of student satisfaction. This can be related to the various features of the LMS

which enhance the ways in which lecturers are able to deliver instruction to their students and monitor progress (for instance auto-marked quizzes for immediate feedback; discussion forums) (Palahicky, 2015). Another factor that is an enabler for using LMSs is that lecturers are able to reach students who could not otherwise access higher education due to a number of reasons.

It has been pointed out that educational technologies such as LMSs facilitate distance education, which in turn “has the ability to reach students who otherwise were not in a position to either attend higher education or continue their education” (Aljabre, 2012, p. 134). This is especially pertinent for KSA which has gender-separated institutions of higher learning and LMSs are proposed as a way of overcoming the severe shortage of female members of faculty (A. Alharbi, 2013).

LMSs can also assist in improving teaching and learning. Culp, Honey, and Mandinach’s (2003) review of two decades of United States educational technology found that the advantages for LMSs include: reaching students when they are not in class (out-of-hours learning and reporting; distance learning); providing opportunities to go beyond classroom materials and gather information for problem-solving and report writing; and broadening the scope of resources available to the class. They also found that technology could help students to pursue their own enquiries and it fulfilled a government requirement for a knowledge-based community. A study conducted by Bernard et al. (2009) similarly found that often course instructors found LMSs methods to be more flexible and engaging than traditional classrooms and found increased student engagement and achievement for asynchronous (offline) courses compared to those that were delivered with either some online content or were face-to-face. However, these advantages are gained when

pedagogical design supports effective implementation, encourages exploratory learning and problem-solving to achieve effective and meaningful learning. These features must also be present for a LMS to be seen as a positive. Given the historical reliance on traditional pedagogy in KSA (see Section 1.4.3) there may need to be a training of pedagogy alongside technology support.

With respect to improving student learning, it has already been noted above that LMSs are associated with a “high student satisfaction level” (Thindwa, 2016, p. 64). The notion that students are more satisfied when they use a LMS implies that the LMS improves their level of learning. This can manifest in the form of increased interactions and exchanges between students, active participation of students in the learning process using LMSs, and the convenience that is associated with accessing and sharing learning materials between lecturers and students (Lonn, Teasley, & Krumm, 2009). In the literature, it has been argued that LMSs “do promote instructional approaches that enhance student learning” (Palahicky, 2015, p. 17). Notably, a LMS like Blackboard provides opportunities for communication through discussion forums, which enhances student interaction and participation in the learning process (Palahicky, 2015). It has also been suggested that technology is effective when the student is “engaged in active, meaningful exercises via technological tools that provide cognitive support” (Schmid et al., 2014, p. 285). All these features are critical for improving student learning, and thus they act as enablers for using LMSs. Students and lecturers alike found Blackboard useful in facilitating learning, and students found that integrating a LMS with traditional learning methods was a significant driver of successful study outcomes (Abanmy & Hussein 2011; Alebaikan 2010; Lee, Hong, & Ling, 2001). Few studies have investigated the efficacy of these tools and their application at universities in the KSA (Alebaikan & Troudi, 2010a; Al-Fahad, 2009).

The use of a LMS is also likely to be encouraged if the LMS aids in improving the working conditions of lecturers. This is due to features such as being able to save time through the use of streamlined communication strategies, enhanced communication with students, and the usefulness of LMSs in relation to teaching activities. The use of streamlined

communication strategies means that a LMS such as Blackboard has the potential to improve the collaborative nature of teaching by enhancing student-lecturer interactions (Coopman, 2009). There are various communication tools in an LMS like Blackboard which lecturers can use to get in touch with their students. These include real time chats,

2015). The use of such tools means, for instance, that a lecturer is able to interact with and monitor the progress of many students without having to meet each of the students face- to-face. LMSs acts as an enabler for the use of such tools. The features that make a LMS useful to lecturers include course delivery tools such as automated testing tools, online marking tools, student tracking and online grade books (Palahicky, 2015). To lecturers, the Blackboard LMS (just like other LMSs) has the potential to improve teaching’s collaborative nature by enhancing student-lecturer interaction experiences (Coopman, 2009). However, a LMS has the potential to make lecturing or teaching a static exercise through an overuse of text (Coopman, 2009). Lecturers, however, have the advantage of being able to update course content, integrate multimedia applications (such as YouTube and blogs), conduct discussions, and initiate or participate in real-time chats with their students (Seechaliao, 2015).

Other features that act as enablers to the use of LMSs include external and internal factors. Internal factors include lecturers’ perceptions regarding the use of LMSs. This includes “their beliefs towards e-learning, and their competence level in using LMS” (Asiri et al., 2012, p. 125). Members of faculty are more likely to use LMSs if they have a positive attitude towards it and if they have the skills required to use the technology (Wichadee, 2015). This point is articulated by Wichadee (2015) based on a review of other studies, which found that “many studies indicate that attitude towards technology are key factors in the adoption and use of technology, specifically an LMS, by faculty” (p. 54). Further, faculty members’ attitudes towards the use or adoption of a technology are affected by the perceived usefulness of that technology (Wichadee, 2015). Also, the perceived usefulness of a

technology has an impact on faculty members’ attitudes towards the use or adoption of that technology (Wichadee, 2015). Given that attitude is related to how an individual responds either favourably or unfavourably towards a given phenomenon (Alshammari, 2015), an individual’s attitude towards a LMS has a bearing on whether they will use it.

Turning to external factors, these are variables that are not within the control of members of faculty. For instance, the fact that the Saudi Arabian government is supportive of

elearning in institutions of higher education (A. Alharbi, 2013) is an enabler to using LMS ssince universities and faculty members will strive to adopt them. In addition, the provision of technical support to faculty members is necessary to ensure successful transition from classroom instruction delivery to online teaching (Alhomod & Shafi, 2013).

A number of research projects have been conducted on LMS in KSA. However, many of these studies (such as Zakaria, Jamal, Bisht, & Koppel, 2013) have focused on students’ perspectives, students' learning and students’ use of LMSs rather than on the experiences of academic staff/lecturers (some exceptions include Al Balawi, 2007). Others (Al Balawi, 2007; Alenezi, 2012; AlMegren & Yassin, 2013; Alqurashi, 2009; K. A. Al-Harbi, 2011) have

indicated how the academic staff influence the adoption of elearning in higher education. Still, there are authors (Albidewi & Tulb, 2014; Al-Shehri, 2010; Guri-Rosenblit, 2005) who observe that institutional factors affect the effective adoption of elearning.

Because of the unique social and cultural situation of KSA, it is important to note the systems of elearning adoption in other societies may not necessarily be applicable in the kingdom, especially when considering gender segregation. In summary, this section has discussed various types of LMSs and how they are used. It has also analysed the situation in KSA regarding the use of LMSs. In doing so, it has identified the barriers to the use of LMSs

• Current issues facing higher education in KSA
• Elearning within higher education
• Elearning within higher education in KSA
• Challenges for elearning delivery in KSA
• Learning management systems (You are here)
• Theoretical Perspective: Technology Acceptance Model
• Research Evaluation
• Enablers for using Blackboard
• Reasons for not using Blackboard in teaching
• Assisting Blackboard use
• External variables
• Perceived usefulness
• Extent of LMS use in KSA universities by female academic staff
• LMS use in higher education in KSA for teaching and learning

Related documents

Systematic Literature Review on Process Mining in Learning Management System

Ieee account.

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

• Open access
• Published: 03 June 2024

Continue nursing education: an action research study on the implementation of a nursing training program using the Holton Learning Transfer System Inventory

• MingYan Shen 1 , 2 &
• ZhiXian Feng 1 , 2  

BMC Medical Education volume  24 , Article number:  610 ( 2024 ) Cite this article

25 Accesses

Metrics details

To address the gap in effective nursing training for quality management, this study aims to implement and assess a nursing training program based on the Holton Learning Transfer System Inventory, utilizing action research to enhance the practicality and effectiveness of training outcomes.

The study involved the formation of a dedicated training team, with program development informed by an extensive situation analysis and literature review. Key focus areas included motivation to transfer, learning environment, and transfer design. The program was implemented in a structured four-step process: plan, action, observation, reflection.

Over a 11-month period, 22 nurses completed 14 h of theoretical training and 18 h of practical training with a 100% attendance rate and 97.75% satisfaction rate. The nursing team successfully led and completed 22 quality improvement projects, attaining a practical level of application. Quality management implementation difficulties, literature review, current situation analysis, cause analysis, formulation of plans, implementation plans, and report writing showed significant improvement and statistical significance after training.

The study confirms the efficacy of action research guided by Holton’s model in significantly enhancing the capabilities of nursing staff in executing quality improvement projects, thereby improving the overall quality of nursing training. Future research should focus on refining the training program through long-term observation, developing a multidimensional evaluation index system, exploring training experiences qualitatively, and investigating the personality characteristics of nurses to enhance training transfer effects.

Peer Review reports

Introduction

The “Medical Quality Management Measures“ [ 1 ] and “Accreditation Standards for Tertiary Hospitals (2020 Edition)” [ 2 ] both emphasize the importance of using quality management tools in medical institutions to carry out effective quality management [ 3 ]. However, there is a notable gap in translating theoretical training into effective, practical application in clinical settings [ 4 ]. This gap is further highlighted in the context of healthcare quality management, as evidenced in studies [ 5 ] which demonstrate the universality of these challenges across healthcare systems worldwide.

Addressing this issue, contemporary literature calls for innovative and effective training methods that transition from passive knowledge acquisition to active skill application [ 6 ]. The Holton Learning Transfer System Inventory [ 7 ] provides a framework focusing on key factors such as motivation, learning environment, and transfer design [ 7 , 8 , 9 ]. This study aims to implement a nursing training program based on the Holton model, using an action research methodology to bridge the theoretical-practical gap in nursing education.

Quality management training for clinical nurses has predominantly been characterized by short-term theoretical lectures, a format that often fails to foster deep engagement and lasting awareness among nursing personnel [ 10 ]. The Quality Indicator Project in Taiwan’s nursing sector, operational for over a decade, demonstrates the effective use of collective intelligence and scientific methodologies to address these challenges [ 11 ]. The proposed study responds to the need for training programs that not only impart knowledge but also ensure the practical application of skills in real-world nursing settings, thereby contributing to transformative changes within the healthcare system [ 12 ].

In April 2021, the Nursing Education Department of our hospital launched a quality improvement project training program for nurses. The initiation of this study is underpinned by the evident disconnect between theoretical training and the practical challenges nurses face in implementing quality management initiatives, a gap also identified in the work [ 13 ]. By exploring the efficacy of the Holton Learning Transfer System Inventory, this study seeks to enhance the practical application of training and significantly contribute to the field of nursing education and quality management in healthcare.

Developing a nursing training program with the Holton Learning Transfer System Inventory

Establishing a research team and assigning roles.

There are 10 members in the group who serve as both researchers and participants, aiming to investigate training process issues and solutions. The roles within the group are as follows: the deputy dean in charge of nursing is responsible for program review and organizational support, integrating learning transfer principles in different settings [ 14 ]; the deputy director of the Nursing Education Department handles the design and implementation of the training program, utilizing double-loop learning for training transfer [ 15 ]; the deputy director of the Nursing Department oversees quality control and project evaluation, ensuring integration of evidence-based practices and technology [ 16 ] and the deputy director of the Quality Management Office provides methodological guidance. The remaining members consist of 4 faculty members possessing significant university teaching experience and practical expertise in quality control projects, and 2 additional members who are jointly responsible for educational affairs, data collection, and analysis. Additionally, to ensure comprehensive pedagogical guidance in this training, professors specializing in nursing pedagogy have been specifically invited to provide expertise on educational methodology.

Current situation survey

Based on the Holton Learning Transfer System Inventory (refer to Fig.  1 ), the appropriate levels of Motivation to Improve Work Through Learning (MTIWL), learning environment, and transfer design are crucial in facilitating changes in individual performance, thereby influencing organizational outcomes [ 17 , 18 ]. Motivation to Improve Work Through Learning (MTIWL) is closely linked to expectation theory, fairness theory, and goal-setting theory, significantly impacting the positive transfer of training [ 19 ]. Learning environment encompasses environmental factors that either hinder or promote the application of learned knowledge in actual work settings [ 20 ]. Transfer design, as a pivotal component, includes training program design and organizational planning.

To conduct the survey, the research team retrieved 26 quality improvement reports from the nursing quality information management system, which were generated by nursing units in 2020. A checklist was formulated, and a retrospective evaluation was conducted across eight aspects, namely, team participation, topic selection feasibility, method accuracy, indicator scientificity, program implementation rate, effect maintenance, and promotion and application. Methods employed in the evaluation process included report analysis, on-site tracking, personnel interviews, and data review within the quality information management system [ 21 ]. From the perspective of motivation [ 22 ], learning environment [ 23 ], and transfer design, a total of 14 influencing factors were identified. These factors serve as a reference for designing the training plan and encompass the following aspects: lack of awareness regarding importance, low willingness to participate in training, unclear understanding of individual career development, absence of incentive mechanisms, absence of a scientific training organization model, lack of a training quality management model, inadequate literature retrieval skills and support, insufficient availability of practical training materials and resources, incomplete mastery of post-training methods, lack of cultural construction plans, suboptimal communication methods and venues, weak internal organizational atmosphere, inadequate leadership support, and absence of platforms and mechanisms for promoting and applying learned knowledge.

Learning Transfer System Inventory

Development of the training program using the 4W1H approach

Drawing upon Holton’s Learning Transfer System Inventory and the hospital training transfer model diagram, a comprehensive training outline was formulated for the training program [ 24 , 25 ]. The following components were considered:

(1) Training Participants (Who): The training is open for voluntary registration to individuals with an undergraduate degree or above, specifically targeting head nurses, responsible team leaders, and core members of the hospital-level nursing quality control team. Former members who have participated in quality improvement projects such as Plan-Do-Check-Act Circle (PDCA) or Quality control circle (QCC) are also eligible.

(2) Training Objectives (Why): At the individual level, the objectives include enhancing the understanding of quality management concepts, improving the cognitive level and application abilities of project improvement methods, and acquiring the necessary skills for nursing quality improvement project. At the team level, the aim is to enhance effective communication among team members and elevate the overall quality of communication. Moreover, the training seeks to facilitate collaborative efforts in improving the existing nursing quality management system and processes. At the operational level, participants are expected to gain the competence to design, implement, and manage nursing quality improvement project initiatives. Following the training, participants will lead and successfully complete a nursing quality improvement project, which will undergo a rigorous audit.

(3) Training Duration (When): The training program spans a duration of 11 months.

(4) Training Content (What): The program consists of 14 h of theoretical courses and 18 h of practical training sessions, as detailed in Table  1 .

(5) Quality Management Approach (How): To ensure quality throughout the training process, two team members are assigned to monitor the entire training journey. This encompasses evaluating whether quality awareness education, quality management knowledge, and professional skills training are adequately covered. Additionally, attention is given to participants’ learning motivation, the emphasis placed on active participation in training methods, support from hospital management and relevant departments, as well as participants’ satisfaction and assessment results. Please refer to Fig.  2 for a visual representation.

In-house training model from Holton Learning Transfer

Implementation of the nursing project training program using the action research method

The first cycle (april 2021).

In the initial cycle, a total of 22 nurses were included as training participants after a self-registration process and qualification review. The criteria used to select these participants, elaborated in Section Development of the training program using the 4W1H approach, ‘Development of the Training Program,’ were meticulously crafted to capture a broad spectrum of experience, expertise, and functional roles within our hospital’s nursing staff. The primary focus was to investigate their learning motivation. The cycle comprised the following key activities:

(1) Training Objectives: The focus was on understanding the learning motivation of the participating nurses.

(2) Theoretical Training Sessions: A total of 7 theoretical training sessions, spanning 14 class hours, were completed. The contents covered various aspects, including an overview of nursing quality improvement projects, methods for selecting project topics, common tools used in nursing quality improvement projects, effective leadership strategies to promote project practices, literature retrieval and evaluation methods, formulation and promotion of project plans, and writing project reports. Detailed course information, including the title, content, and class hours, is listed in Table  1 . At the end of each training session, a course satisfaction survey was conducted.

(3) Assessment and Reporting: Following the completion of the 7 training sessions, a theoretical assessment on quality management knowledge was conducted. Additionally, nurses were organized to present their plans for special projects to be carried out during the training. Several issues were identified during this cycle:

Incomplete Literature Review Skills: Compared to other quality control tools, nursing quality improvement project places more emphasis on the scientific construction of project plans. The theoretical evaluation and interviews with nurses highlighted the incomplete and challenging nature of their literature review skills.

Insufficient Leadership: Among the participants, 6 individuals were not head nurses, which resulted in a lack of adequate leadership for their respective projects.

Learning environment and Support: The learning environment, as well as the support from hospital management and relevant departments, needed to be strengthened.

Second cycle (may-october 2021)

In response to the issues identified during the first cycle, our approach in the second cycle was both corrective and adaptive, focusing on immediate issues while also setting the stage for addressing any emerging challenges. The team members actively implemented improvements during the second cycle. The key actions taken were as follows:

(1) Establishing an Enabling Organizational Environment: The quality management department took the lead, and multiple departments collaborated in conducting the “Hospital Safety and Quality Red May” activity. This initiative aimed to enhance the overall quality improvement atmosphere within the hospital. Themed articles were also shared through the hospital’s WeChat public account.

(2) Salon-style Training Format: The training sessions were conducted in the form of salons, held in a meeting room specifically prepared for this purpose. The room was arranged with a round table, warm yellow lighting, green plants, and a coffee bar, creating a conducive environment for free, democratic, and equal communication among the participants. The salon topics included revising project topic selection, conducting current situation investigations, facilitating communication and guidance for literature reviews, formulating improvement plans, implementing those plans, and writing project reports. After the projects were presented, quality management experts provided comments and analysis, promoting the transformation of training outcomes from mere memory and understanding to higher-level abilities such as application, analysis, and creativity.

(3) Continuous Support Services: Various support services were provided to ensure ongoing assistance. This included assigning nursing postgraduates to aid in literature retrieval and evaluation. Project team members also provided on-site guidance and support, actively engaging in the project improvement process to facilitate training transfer.

(4) Emphasis on Spiritual Encouragement: The Vice President of Nursing Department actively participated in the salons and provided feedback on each occasion. Moreover, the President of the hospital consistently commended the training efforts during the weekly hospital meetings.

Issues identified in this cycle

(1) Inconsistent Ability to Write Project Documents: The proficiency in writing project documents for project improvement varied among participants, and there was a lack of standardized evaluation criteria. This issue had the potential to impact the quality of project dissemination.

(2) Lack of Clarity Regarding the Platform and Mechanism for Training Result Transfer: The platform and mechanisms for transferring training results were not clearly defined, posing a challenge in effectively sharing and disseminating the outcomes of the training.

The third cycle (November 2021-march 2022)

During the third cycle, the following initiatives were undertaken.

(1) Utilizing the “Reporting Standards for Quality Improvement Research (SQUIRE)”, as issued by the US Health Care Promotion Research, to provide guidance for students in writing nursing project improvement reports.

(2) Organizing a hospital-level nursing quality improvement project report meeting to acknowledge and commend outstanding projects.

(3) Compiling the “Compilation of Nursing Quality Improvement Projects” for dissemination and exchange among nurses both within and outside the hospital.

(4) Addressing the issue of inadequate management of indicator monitoring data, a hospital-level quality index management platform was developed. The main evaluation data from the 22 projects were entered into this platform, allowing for continuous monitoring and timely intervention.

Effect evaluation

To assess the efficacy of the training, a diverse set of evaluation metrics, encompassing both outcome and process measures [ 26 ]. These measures can be structured around the four-level training evaluation framework proposed by Donald Kirkpatrick [ 27 ].

Process evaluation

Evaluation method.

To assess the commitment and support within the organization, the process evaluation involved recording the proportion of nurses’ classroom participation time and the presence of leaders during each training session. Additionally, a satisfaction survey was conducted after the training to assess various aspects such as venue layout, time arrangement, training methods, lecturer professionalism, content practicality, and interaction. On-site recycling statistics were also collected for project evaluation purposes.

Evaluation results the results of the process evaluation are as follows

Nurse training participation rate: 100%.

Training satisfaction rate (average): 97.75%.

Proportion of nurses’ participation time in theoretical training sessions (average): 36.88%.

Proportion of nurses’ participation time in salon training sessions (average): 74.23%.

Attendance rate of school-level leaders: 100%.

Results evaluation

Assessment of theoretical knowledge of quality management.

To evaluate the effectiveness in enhancing the trainees’ theoretical knowledge of quality management, the research team conducted assessments before the training, after the first round of implementation, and after the third round of implementation. Assessments to evaluate the effectiveness of the training program were conducted immediately following the first round of implementation, and after the third round of implementation. This dual-timing approach was designed to evaluate both the immediate impact of the training and its sustained effects over time, addressing potential influences of memory decay on the study results. The assessment consisted of a 60-minute examination with different question types, including 30 multiple-choice questions (2 points each), 2 short-answer questions (10 points each), and 1 comprehensive analysis question (20 points). The maximum score achievable was 100 points.

The assessment results are as follows:

Before training (average): 75.05 points.

After the first round of implementation (average): 82.18 points.

After the third round of implementation (average): 90.82 points.

Assessment of difficulty in quality management project implementation

To assess the difficulty of implementing quality management projects, the trainees completed the “Quality Management Project Implementation Difficulty Assessment Form” before and after the training. They self-evaluated 10 aspects using a 5-point scale, with 5 indicating the most difficult and 1 indicating no difficulty. The evaluation results before and after implementation are presented in Table  2 .

Statistically significant differences were found in the following items: literature review, current situation analysis, cause analysis, plan formulation, implementation plan, and report writing. This indicates that the training significantly enhanced the nurses’ confidence and ability to tackle practical challenges.

Evaluation of transfer effect

To assess how effectively the training translated into practical applications. The implementation of the 22 quality improvement projects was evaluated using the application hierarchy analysis table. The specific results are presented in Table  3 .

In addition, the “Nursing Project Guidance Manual” and “Compilation of Nursing Project Improvement Projects” were compiled and distributed to the hospital’s management staff, nurses, and four collaborating hospitals, receiving positive feedback. The lecture titled “Improving Nurses’ Project Improvement Ability Based on the Training Transfer Theory Model” shared experiences with colleagues both within and outside the province in national and provincial teaching sessions in 2022. Furthermore, four papers were published on the subject.

The effectiveness of the training program based on the Holton Learning transfer System Inventory

The level of refined management in hospitals is closely tied to the quality management awareness and skills of frontline medical staff. Quality management training plays a crucial role in improving patient safety management and fostering a culture of quality and safety. Continuous quality improvement is an integral part of nursing management, ensuring that patients receive high-quality and safe nursing care. Compared to the focus of existing literature on the individual performance improvements following nursing training programs [ 28 , 29 , 30 ], our study expands the evaluation framework to include organizational performance metrics. Our research underscores a significantly higher level of organizational engagement as evidenced by the 100% attendance rate of school-level leaders. The publication of four papers related to this study highlights not only individual performance achievements but also significantly broadens the hospital organization’s impact on quality management, leading to meaningful organizational outcomes.

Moreover, our initiative to incorporate indicators of quality projects into a hospital-level evaluation index system post-training signifies a pivotal move towards integrating quality improvement practices into the very fabric of organizational operations. In training programs, it is essential not only to achieve near-transfer, but also to ensure that nurses continuously apply the acquired management skills to their clinical work, thereby enhancing quality, developing their professional value, and improving organizational performance. The Holton learning Transfer System Inventory provides valuable guidance on how to implement training programs and evaluate their training effect.

This study adopts the training transfer model as a framework to explore the mechanisms of “how training works” rather than simply assessing “whether training works [ 31 ].” By examining factors such as Motivation to Improve Work Through Learning (MTIWL), learning environment, and transfer design, the current situation is analyzed, underlying reasons are identified, and relevant literature is reviewed to develop and implement training programs based on the results of a needs survey. While individual transfer motivation originates from within the individual, it is influenced by the transfer atmosphere and design. By revising the nurse promotion system and performance management system and aligning them with career development, nurses’ motivation to participate and engage in active learning has significantly increased [ 32 ]. At the learning environment level, enhancing the training effect involves improving factors such as stimulation and response that correspond to the actual work environment [ 33 ]. This project has garnered attention and support from hospital-level leaders, particularly the nursing dean who regularly visits the training site to provide guidance, which serves as invaluable recognition. Timely publicity and recognition of exemplary project improvement initiatives have also increased awareness and understanding of project knowledge among doctors and nurses, fostering a stronger quality improvement atmosphere within the team.

Transfer design, the most critical component for systematic learning and mastery of quality management tools, is achieved through theoretical lectures, salon exchanges, and project-based training. These approaches allow nurses to gain hands-on experience in project improvement under the guidance of instructors. Throughout the project, nurses connect project management knowledge and skills with practical application, enabling personal growth and organizational development through problem-solving in real work scenarios. Finally, a comprehensive evaluation of the training program was conducted, including assessments of theoretical knowledge, perception of management challenges, and project quality. The results showed high satisfaction among nurses, with a satisfaction rate of 97.75%. The proportion of nurses’ participation time in theoretical and practical training classes was 36.88% and 74.23%, respectively. The average score for theoretical knowledge of quality management increased from 75.05 to 90.82. There was also a significant improvement in the evaluation of the implementation difficulties of quality management projects. Moreover, 22 nurses successfully led the completion of one project improvement project, with six projects focusing on preventing the COVID-19 pandemic, demonstrating valuable crisis response practices.

Action research helps to ensure the quality of organizational management of training

Well-organized training is the basis for ensuring the scientific and standardized development of nursing project improvement activities. According to the survey results of the current situation, there is a lot of room for improvement in the training quality; since it is the first time to apply the Holton training transfer model to the improvement training process of nurses in the hospital, in order to allow the nurses to have sufficient time to implement and evaluate the improvement project, the total training time Set at 11 months, a strong methodology is required to ensure training management during this period. Action research is a research method that closely combines research with solving practical problems in work. It is a research method aimed at solving practical problems through self-reflective exploration in realistic situations, emphasizing the participation of researchers and researchees. Practice, find problems in practice, and adjust the plan in a timely manner. According to the implementation of the first round, it was found that nurses had insufficient literature review skills, insufficient leadership, and lack of support from hospital management and related departments [ 32 ]. In the second round, the training courses were carried out in the form of salons. The project team members went deep into the project to improve on-site guidance, arranged graduate students to assist in document retrieval and evaluation, and promoted the transfer of training; the “Hospital Safety and Quality Red May” activity was carried out, and the vice president of nursing Regularly participate in the salon and make comments. The problems exposed after this round of implementation are the low quality of the project improvement project document, and the unclear platform and mechanism for the transfer of training results. In the third round, the “Reporting Standards for Quality Improvement Research (SQUIRE)” was used to standardize the writing of the report [ 33 ], and the “Compilation of Nursing Project Improvement Projects” was completed, and the main evaluation data of 22 projects were entered into the hospital-level quality index management platform for continuous monitoring and intervention. As of May 2022, the effect maintenance data of each project has reached the target value. It can not only produce useful improvement projects, but also help to promote the dissemination and penetration of quality awareness.

Future research directions

Drawing on the Holton training evaluation model, this study implemented nurse quality improvement project training using action research methodology, resulting in a successful exploration practice, and achieving positive transfer effects. To further advance this research area, the following future research directions are recommended:

Summarize the experiences gained from this action research training and continue to refine and enhance the training program. Through ongoing practice, reflection, and refinement in subsequent training sessions, long-term observation of the transfer effects can be conducted to establish an effective experiential model that can serve as a reference for future initiatives.

Develop a multidimensional evaluation index system for assessing transfer effects. A comprehensive framework that captures various dimensions of transfer, such as knowledge application, skill utilization, and behavior change, should be established. This will enable a more holistic and accurate assessment of the training program’s impact on the participants and the organization.

Conduct qualitative research to explore the training experiences of nurses. By gathering in-depth insights through interviews or focus group discussions, a deeper understanding of the nurses’ perceptions, challenges, and facilitators of training transfer can be obtained. This qualitative exploration will provide valuable information to further refine and tailor the training program to meet the specific needs and preferences of the nurses.

Investigate the personality characteristics of nurses who actively engage in training transfer and consider developing them as internal trainers. By identifying and cultivating nurses with a proactive attitude and a strong inclination towards knowledge application and skill development, the organization can enhance employee participation and initiative. These internal trainers can play a crucial role in motivating their colleagues and driving the transfer of training outcomes into daily practice.

By pursuing these future research directions, the field of healthcare and nursing care can continue to advance in optimizing training programs, enhancing transfer effects, and ultimately improving the quality of care and patient outcomes.

Limitations

The research was conducted with a cohort of 22 nurses and a 10-member research team from Grade 3, Class A hospitals in China Southeast. This specific composition and the relatively small sample size may affect the generalizability of our findings. The experiences and outcomes observed in this study might not fully encapsulate the diverse challenges and environments encountered by nursing professionals in varying healthcare settings. The significant improvements noted in the capabilities of the participating nursing staff underscore the potential impact of the training program. However, the study’s focus on a specific demographic—nurses from high-grade hospitals in a developed urban center—may limit the external validity of the findings.

Conclusions

This study affirms the efficacy of the Holton Learning Transfer System Inventory-based training program, coupled with action research, in significantly advancing nursing quality management practices. The strategic incorporation of motivation to improve work through learning, an enriched learning environment, and thoughtful transfer design significantly boosted the nurses’ engagement, knowledge acquisition, and practical application of quality management tools in their clinical work.

It highlights the importance of continuous learning, organizational support, and methodological flexibility in achieving sustainable improvements in healthcare quality and safety. Future endeavors should aim to expand the scope of this training model to diverse nursing contexts and evaluate its long-term impact on organizational performance and patient care outcomes.

Data availability

The datasets generated and/or analyzed during the current study are not publicly available due to hospital policy but are available from the corresponding author on reasonable request.

National Health and Family Planning Commission. Medical Quality Management Measures. Published 2016. Accessed September 19, 2018. http://www.nhc.gov.cn/fzs/s3576/201610/ae125f28eef24ca7aac57c8ec530c6d2.shtml .

National Health Commission of the People’s Republic of China. Notice of the National Health Commission on Printing and Distributing the Standards for the Review of Third-level Hospitals (2020 Edition) [EB/OL]. (2021-10-21) [2022-07-17].

Grossu-Leibovica D, Kalkis H. Total quality management tools and techniques for improving service quality and client satisfaction in the healthcare environment: a qualitative systematic review. Manage Sci Lett. 2023;13(2):118–23. https://doi.org/10.1051/shsconf/202213102009 .

Article   Google Scholar  

Shade L, Reeves K, Rees J, Hendrickson L, Halladay J, Dolor RJ, Bray P, Tapp H. Research nurses as practice facilitators to disseminate an asthma shared decision making intervention. BMC Nurs. 2020;19:40. https://doi.org/10.1186/s12912-020-00414-0 .

Ali J, Jusoh A, Abbas A, Nor K. Global trends of Service Quality in Healthcare: a bibliometric analysis of Scopus Database. J Contemp Issues Bus Government. 2021;27:2917.

Google Scholar  

Ahmed FA, Choudhary RA, Khan H, Ayub F, Hassan SSU, Munir T, Asif F, Ajani K, Jaffer M, Tharani Z, Aboumatar HJ, Haider A, Latif A. Incorporating Patient Safety and Quality Course into the nursing curriculum: an Assessment of Student gains. J Patient Saf. 2023;19(6):408–14. https://doi.org/10.1097/PTS.0000000000001146 .

Holton EF. Holton’s evaluation model: new evidence and construct elaborations. Adv Developing Hum Resour. 2005;7(1):37–54. https://doi.org/10.1177/1523422304272080 .

Rigot SK, DiGiovine KM, Boninger ML, Hibbs R, Smith I, Worobey LA, Connerton C, Mason J, Bonhotal S. (2023). Peer-Led Functional Mobility and Transfer Training. Nurse Educator, 48(5), 286.

Wang C, Liu A, Xu J, et al. Work Method and Effect of Hospital Quality Management Circle Stage Management model [J]. Chin Hosp. 2016;20(12):23–5.

Elshama S. Quality Management in Medical Education between Theory and Application: paradigm shift or falsification of. J Clin Case Rep Stud. 2022;3:1–5. https://doi.org/10.31579/2690-8808/109 .

Chang S-J, Huang HH-C, Li-Hua, Chang H. Taiwan quality indicator project and hospital productivity growth. Omega. 2011;39(1):14–22.

Arnold AP, Laurene Finley [email protected], Roberta G, Sands, Joretha Bourjolly & Victoria Stanhope. (2012) Training Mental Health Providers in Cultural Competence: A Transformative Learning Process, American Journal of Psychiatric Rehabilitation, 15:4, 334–356, https://doi.org/10.1080/15487768.2012.733287 .

Chang S-J, Hsiao H-C, Huang L-H, Chang H. 2011. Taiwan quality indicator project and hospital productivity growth, Omega, Elsevier, vol. 39(1), pages 14–22, January. 2010.01.006.

Finn F, Chesser-Smyth P. Promoting learning transfer in Preceptor Preparation. J Nurses Prof Dev. 2013;29:309–15. https://doi.org/10.1097/NND.0000000000000014 .

Guzman G, Fitzgerald J, Fulop L, Hayes K, Poropat A, Avery M, Campbell S, Fisher R, Gapp R, Herington C, McPhail R, Vecchio N. How best practices are copied, transferred, or translated between health care facilities: a conceptual framework. Health Care Manage Rev. 2015;40:193–202. https://doi.org/10.1097/HMR.0000000000000023 .

Billings D, Connors H, Skiba D. Benchmarking Best practices in web-based nursing courses. Adv Nurs Sci. 2001;23:41–52. https://doi.org/10.1097/00012272-200103000-00005 .

Devos C, Dumay X, Bonami M, Bates R, Holton E. The learning transfer system inventory (LTSI) translated into French: Internal structure and predictive validity. Eur Economics: Labor Social Conditions eJournal. 2007. https://doi.org/10.1111/j.1468-2419.2007.00280.x .

Mongkolsirikiet K, Akaraborworn C, Research Model. (2019). A Revisit of Holton’s HRD Evaluation and (2005) for Learning Transfer., 12, 15–34. https://doi.org/10.14456/JCDR-HS.2019.12 .

Yaqub Y, Singh A. Impact of training design on trainees’ motivation: an empirical study. Industrial Commercial Train. 2021. https://doi.org/10.1108/ict-05-2021-0038 .

Dewettinck K, Dijk H. Linking Belgian employee performance management system characteristics with performance management system effectiveness: exploring the mediating role of fairness. Int J Hum Resource Manage. 2013;24:806–25. https://doi.org/10.1080/09585192.2012.700169 .

Duprez V, Vandecasteele T, Verhaeghe S, Beeckman D, Hecke A. The effectiveness of interventions to enhance self-management support competencies in the nursing profession: a systematic review. J Adv Nurs. 2017;73:1807–24. https://doi.org/10.1111/jan.13249 .

Norouzi S, Mogadam F. Experiences of nursing Student\‘s clinical evaluation: a qualitative content analysis. J Med Educ Dev. 2016;11:134–45.

Niskala J, Kanste O, Tomietto M, Miettunen J, Tuomikoski A, Kyngäs H, Mikkonen K. Interventions to improve nurses’ job satisfaction: a systematic review and meta-analysis. J Adv Nurs. 2020. https://doi.org/10.1111/jan.14342 .

Gkioka M, Schneider J, Kruse A, Tsolaki M, Moraitou D, Teichmann B. Evaluation and Effectiveness of Dementia Staff Training Programs in General Hospital settings: a narrative synthesis with Holton’s three-level Model Applied. J Alzheimers Dis. 2020;78:1089–108. https://doi.org/10.3233/JAD-200741 .

Ghazvini A, Shukur Z. Awareness training transfer and Information Security Content Development for Healthcare Industry. Int J Adv Comput Sci Appl. 2016. https://doi.org/10.14569/IJACSA.2016.070549 . 7.

Ragsdale J, Berry A, Gibson J, Herber-Valdez C, Germain L, Engle D. Evaluating the effectiveness of undergraduate clinical education programs. Med Educ Online. 2020;25. https://doi.org/10.1080/10872981.2020.1757883 .

Kirkpatrick DL. Techniques for evaluation training programs. J Am Soc Train Dir. 1959;13:21–6.

Kirkman TR. High Fidelity Simulation Effectiveness in Nursing Students’ Transfer of Learning International Journal of Nursing Education Scholarship, vol. 10, no. 1, 2013, pp. 171–176. https://doi.org/10.1515/ijnes-2012-0009 .

Chen SL, Huang TW, Liao IC, Liu C. Development and validation of the simulation learning effectiveness inventory. J Adv Nurs. 2015;71(10):2444–53. https://doi.org/10.1111/jan.12707 .

Ayed A, Khalaf I, Fashafsheh I, Saleh A, Bawadi H, Abuidhail J, Thultheen I, Joudallah H. Effect of High-Fidelity Simulation on Clinical Judgment among nursing students. Inquiry: J Med Care Organ Provis Financing. 2022;59. https://doi.org/10.1177/00469580221081997 .

Yun J, Kim D, Park Y. The influence of informal learning and learning transfer on nurses’ clinical performance: a descriptive cross-sectional study. Nurse Educ Today. 2019. https://doi.org/10.1016/J.NEDT.2019.05.027 .

Bhatti M, Ali S, Isa M, Battour M. Training transfer and transfer motivation: the influence of individual, environmental, situational, Training Design, and affective reaction factors. Perform Improv Q. 2014;27:51–82. https://doi.org/10.1002/PIQ.21165 .

Kontoghiorghes C. Factors affecting training effectiveness in the context of the introduction of a New Technology–A U.S. Case Study. Int J Train Dev. 2001;5:248–60. https://doi.org/10.1111/1468-2419.00137 .

Download references

This study was funded by Department of Education of Zhejiang Province, Grant Number jg20220475.

Author information

Authors and affiliations.

School of Nursing, Zhejiang Shuren University, 8 Shuren Road, 310015, Hangzhou, ZheJiang, China

MingYan Shen & ZhiXian Feng

Department of Nursing, Shulan (Hangzhou) Hospital, Shulan International Medical College, Zhejiang Shuren University, 310022, Hangzhou, China

You can also search for this author in PubMed   Google Scholar

Contributions

The following statements specify the individual contributions of each author to the manuscript titled “Continue Nursing Education: An Action Research Study on the Implementation of a Nursing Training Program Using the Holton Learning Transfer System Inventory”:ZhiXian Feng conceived and designed the analysis; led the research team and coordinated the project; critically reviewed and revised the manuscript for important intellectual content; oversaw the implementation of the training program; MingYan Shen conducted the research; collected and organized the data; analyzed and interpreted the data; contributed to the statistical analysis; wrote the initial draft of the manuscript; managed logistics and operational aspects of the study.

Corresponding author

Correspondence to ZhiXian Feng .

Ethics declarations

Ethics approval and consent to participate.

Approval of this study was granted by the Research Ethics Committee of Shulan Hospital (Approval no. KY2021042).

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Prior to participation, all subjects (or their legal guardians) were informed about the nature, objectives, potential benefits, and risks of the study. Written informed consent was obtained from all subjects and/or their legal guardians. All data were collected and processed maintaining strict confidentiality and anonymity, safeguarding the privacy and rights of all participants.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Shen, M., Feng, Z. Continue nursing education: an action research study on the implementation of a nursing training program using the Holton Learning Transfer System Inventory. BMC Med Educ 24 , 610 (2024). https://doi.org/10.1186/s12909-024-05552-6

Download citation

Received : 16 December 2023

Accepted : 13 May 2024

Published : 03 June 2024

DOI : https://doi.org/10.1186/s12909-024-05552-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Nursing education
• Quality improvement
• Action research
• Holton Learning Transfer System Inventory

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Comprehensive analysis of digital twins in smart cities: a 4200-paper bibliometric study

• Open access
• Published: 27 May 2024
• Volume 57 , article number  154 , ( 2024 )

Cite this article

You have full access to this open access article

• Rasha F. El-Agamy   ORCID: orcid.org/0009-0005-0519-3870 1 , 2 ,
• Hanaa A. Sayed   ORCID: orcid.org/0000-0003-0728-6323 1 , 3 ,
• Arwa M. AL Akhatatneh   ORCID: orcid.org/0009-0009-2133-1822 1 ,
• Mansourah Aljohani   ORCID: orcid.org/0000-0001-5233-7738 1 &
• Mostafa Elhosseini   ORCID: orcid.org/0000-0002-1259-6193 1 , 4  

631 Accesses

Explore all metrics

This survey paper comprehensively reviews Digital Twin (DT) technology, a virtual representation of a physical object or system, pivotal in Smart Cities for enhanced urban management. It explores DT's integration with Machine Learning for predictive analysis, IoT for real-time data, and its significant role in Smart City development. Addressing the gap in existing literature, this survey analyzes over 4,220 articles from the Web of Science, focusing on unique aspects like datasets, platforms, and performance metrics. Unlike other studies in the field, this research paper distinguishes itself through its comprehensive and bibliometric approach, analyzing over 4,220 articles and focusing on unique aspects like datasets, platforms, and performance metrics. This approach offers an unparalleled depth of analysis, enhancing the understanding of Digital Twin technology in Smart City development and setting a new benchmark in scholarly research in this domain. The study systematically identifies emerging trends and thematic topics, utilizing tools like VOSviewer for data visualization. Key findings include publication trends, prolific authors, and thematic clusters in research. The paper highlights the importance of DT in various urban applications, discusses challenges and limitations, and presents case studies showcasing successful implementations. Distinguishing from prior studies, it offers detailed insights into emerging trends, future research directions, and the evolving role of policy and governance in DT development, thereby making a substantial contribution to the field.

Similar content being viewed by others

Data Science and Analytics: An Overview from Data-Driven Smart Computing, Decision-Making and Applications Perspective

Smart tourism: foundations and developments.

The Digital Twin of the City of Zurich for Urban Planning

Avoid common mistakes on your manuscript.

1 Introduction

Digital Twin (DT) technology, a cornerstone of the Industry 4.0 era, represents a significant paradigm shift in how we interact with and understand physical systems and assets. Originating from Grieve's 2002 lecture at the University of Michigan (Grieves 2005 ) and later refined by NASA in 2010 (NASA 2010 ), the concept of DT has evolved into a sophisticated, multi-faceted approach to simulation and analysis. A Digital Twin is broadly defined as a digitally created virtual model of a physical object that leverages data to emulate the real-world behavior of the physical entity. It facilitates interaction and interoperability between the physical and virtual entities through interactive feedback, data integration, analysis, and iterative decision-making for optimized control, safety monitoring, and data analysis (Stark et al. 2017 ; Rosen et al. 2015 ).

Kritzinger et al. (Kritzinger et al. 2018 ) further categorized DT into three subtypes: digital model, digital shadow, and digital twin, each representing varying degrees of interaction and correlation between the physical and digital states.

The structure of a DT encompasses hardware and software components connected via middleware. The hardware typically includes IoT sensors and actuators, with the middleware playing a critical role in data management and communication between hardware and software. The software component, often an analytics engine, utilizes machine learning algorithms to transform raw data into actionable insights (Kritzinger et al. 2018 ). As depicted in Fig.  1 , this system encompasses the various components constituting the digital twin architecture.

Digital twin system structure. This diagram illustrates the essential components of a Digital Twin system, showcasing hardware with IoT sensors, middleware for data management, and a software analytics engine

Before going into the various applications of digital twins in various industries, it is important to comprehend the nature of digital twins and two closely connected ideas: digital model and digital shadow. These concepts are crucial for understanding this technology's depth (Kritzinger et al. 2018 ). The first concept is the Digital Model, a static digital representation of a physical object without automatic data exchange between the physical and digital entities. It can take many forms, including simulations, CAD files, 3D models, and mathematical algorithms. Digital models help design, optimize, and test by enabling the visualization, analysis, and manipulation of objects or systems in a digital context. A model is typically an estimation or prediction of how a system, process, or physical thing could function in the future or a certain setting. The second concept is Digital Shadow, which represents unidirectional information flow from the physical object to its digital counterpart, reflecting changes in the physical object. Through sensors, Internet of Things (IoT) devices, or other means, digital shadow gathers data from the asset (a database, a railroad system, or a banking platform). It delivers information that is fed into the model. This indicates that a digital shadow is current with the real object. It is helpful to understand it because it accurately depicts the asset in enough detail. The last concept is the Digital Twin, a dynamic, interactive digital representation capable of simulating, predicting, and interacting with data, showing a reciprocal impact between physical and digital states. Digital twins help with analysis, optimization, and predictive maintenance by simulating, monitoring, and controlling real-world systems or objects. They provide insights for enhancing effectiveness, dependability, and performance, as well as live feedback loops.

Figure  2 demonstrates the evolution from a basic digital model, lacking interactive data exchange, to a digital twin that dynamically mirrors and interacts with its physical counterpart, allowing for a two-way flow of information and continuous adaptation.

The progression from a static digital model to a dynamic digital twin, emphasizing reciprocal interaction between physical and digital assets

Digital twins' ability to reproduce physical items, processes, and systems in a virtual environment makes them useful in various applications. This technology has applications across various sectors and domains, providing several benefits and chances for innovation. For example, in industry, they are used for predictive maintenance, optimizing energy usage in smart buildings, and simulating traffic patterns in smart cities.

In the IoT sector, DTs are pivotal, acting as a critical bridge between the physical and digital realms. They allow for the seamless integration of digital and physical entities, enhancing maintenance capabilities and improving equipment performance monitoring (Fang et al. 2022 ; Mihai et al. 2022 ; Hinchy et al. 2019 ; Guo 2020; Wang and Luo 2021 ; Rajesh et al. 2019 ; Revetria et al. 2019 ). IoT can be seen as the vehicle that drives data to Digital Twins, enabling these virtual entities to replicate and interact with their physical counterparts in real-time. Digital Twins depend heavily on IoT technologies for data acquisition. IoT devices like sensors, RFID tags, and smart wearables collect data from the physical environment, which the digital twin then utilizes for various analyses. This data integration facilitated by IoT is crucial for applications ranging from predictive maintenance in industrial settings to real-time monitoring and augmented reality applications (Rajesh et al. 2019 ; Revetria et al. 2019 ). As described in sources monitoring (Fang et al. 2022 ; Mihai et al. 2022 ; Hinchy et al. 2019 ; Guo et al. 2020 ; Wang and Luo 2021 ; Rajesh et al. 2019 ; Revetria et al. 2019 ), IoT's role is not just about data collection but also about ensuring seamless integration of physical and virtual worlds, thus forming the backbone of any DT system.

While the Internet of Things (IoT) plays a major role in shaping and augmenting the capabilities of digital twins, machine learning augments these capabilities by allowing digital twins to analyze data, forecast, identify anomalies, optimize performance, customize experiences, and learn and improve continuously. Integrating machine learning with DT technology enables real-time, autonomous analysis of extensive data streams, enhancing decision-making and optimizing asset and system performance (Rathore et al. 2021 ; Dong et al. 2019 ; Zohdi 2020 ; Jaensch et al. 2018 ; He et al. 2019 ). Machine Learning, a pivotal branch of Artificial Intelligence, involves algorithms that enable systems to learn and adapt from data without being explicitly programmed. Its relationship with Digital Twin technology is synergistic. Digital Twins, virtual replicas of physical entities, systems, or environments, require advanced analytical capabilities to process and interpret the vast amount of data they receive. This is where Machine Learning comes into play. Machine Learning algorithms in DT systems facilitate the autonomous, real-time analysis of extensive data streams. These algorithms are adept at detecting patterns, making predictions, and optimizing processes based on the data ingested from the physical assets that the digital twins mirror. For instance, Rathore et al. 2021 (Rathore 2021) highlighted how applying advanced AI techniques to data within a DT system enables the creation of an 'intelligent' digital twin. This intelligence is manifested in capabilities like predictive maintenance, operational optimization, and dynamic decision-making based on a continuous stream of sensor and virtual data. The application of various machine learning models, such as Deep Neural Networks (DNNs) or Genetic Algorithms (GAs), is contingent upon the specific requirements and use cases of the intended digital twins (Dong et al. 2019 ; Zohdi 2020 ; Jaensch 2018; He et al. 2019 ). Therefore, Machine Learning is not just a complementary technology for Digital Twins but a fundamental enabler of their advanced functionalities.

In the sector of smart cities, DTs are used for urban planning, traffic management, environmental monitoring, energy management, waste management, public safety, infrastructure maintenance, water management, healthcare, public services, tourism, citizen engagement, economic development, and climate resilience. They provide real-time data crucial for emergency response, optimizing public transportation, and ensuring efficient city operations (Allam and Jones 2021 ; Bouzguenda et al. 2019 ; Svítek et al. 2019 ; Yu et al. 2021 ; Ghosh et al. 2016 ). Smart Cities represent urban areas that integrate various electronic data collection sensors to manage assets, resources, and services efficiently. Digital Twins, within the context of Smart Cities, act as sophisticated tools for urban planning, management, and enhancement of living conditions. They utilize data gathered via IoT devices and analyze it using machine learning algorithms to optimize city operations and decision-making processes. Besides, they contribute to traffic management, environmental monitoring, energy distribution, public safety, and more (Allam and Jones 2021 ; Bouzguenda et al. 2019 ; Svítek et al. 2019 ; Yu et al. 2021 ; Ghosh et al. 2016 ). For example, digital twins utilize data from sensors and cameras to optimize traffic flow and public transportation systems in traffic and transportation management. They use real-time data to monitor air and water quality in environmental monitoring. In energy management, digital twins aid in the operation of smart grids and in identifying potential energy conservation areas. These applications underline the comprehensive impact that Digital Twins, empowered by IoT and ML, can have in transforming urban environments into more efficient, sustainable, and responsive entities.

The primary aim of this paper is to engage in a comprehensive bibliometric analysis, examining the evolving landscape of Digital Twin technology within Smart Cities. The study is dedicated to methodically examining the scholarly dialogue, identifying predominant trends, and revealing key themes and collaborative networks in this area. We aim to provide a detailed, structured understanding of Digital Twin technology's role in urban development, filling a notable void in existing literature reviews. The survey's distinctiveness stems from its thorough data-gathering approach for bibliometric analysis in the field of Digital Twin technology and Smart Cities, selecting the Web of Science database for its broad interdisciplinary coverage and meticulously filtering over 4,220 pertinent articles, enhancing the depth and scope of analysis in these domains.

Our research found that various publications in various literary works are advancing the DT idea. Because there are so many articles available, academics have also published several survey papers that aim to review the current state-of-the-art in digital transformation (DT) development, inform other innovators about potential research gaps, questions, and directions, and point the industry toward potential DT use cases that could yield substantial business value in their particular domain.

Current literature predominantly concentrates on applying digital twin technology within specific facets of smart cities. For instance, Jafari et al. ( 2023 ) and He et al. ( 2023 ) explore the utilization of digital twin (DT) technology in enhancing various sectors of energy management within smart cities, encompassing transportation systems, power grids, and microgrids. Weil (2023) delves into the infrastructure elements of digital twins in smart cities, focusing on storage, computation, and network components. Nica et al. ( 2023 ) investigates Multi-Sensor Fusion Technology's role in sustainable urban governance networks. Dani et al. ( 2023 ) introduces an architectural framework underpinning the flow for digital twin platform development aimed at urban condition monitoring. Lam et al. ( 2023 ) outlines a use case for the 3D visualization of a smart village in Busan, South Korea, employing a 3D Geospatial platform. Paripooranan et al. ( 2020 ) suggests augmented reality (AR)-assisted DT as a pioneering approach towards the future transformation of human-centric industries. Mora (2023) highlights the importance of incorporating innovation management theories into the exploration of smart city transitions, offering novel insights and practical approaches to enhance the governance of smart cities through an innovation management lens. Ariyachandra and Wedawatta ( 2023 ) provides an overview of digital twin technologies' implications on disaster risk management, addressing the challenges of implementing digital twins in smart cities. Additionally, several reviews, including those by Weil (2023) and Wang (2024), focus on bibliometric analyses concerning digital twins in the realm of smart cities.

This work aims to support the other existing survey initiatives and provide a comprehensive comprehension of the DT. The paper gives an in-depth overview of the DT idea, architecture, enabling technologies, applications in smart cities, challenges, performance metrics, datasets, software, and use cases for deploying DTs in diverse industries, complementing prior research. This paper aims to fill a critical gap in understanding the expansive and evolving field of Digital Twin technology and its integration into Smart City development. This study is driven by the need to systematically synthesize and analyze the burgeoning body of research in this interdisciplinary area, providing clarity and direction for future studies. This survey's uniqueness and unprecedented nature stem from its comprehensive and systematic bibliometric analysis of over 4,220 articles on Digital Twin technology and Smart Cities. A focused examination of specialized areas such as datasets, platforms, and performance metrics marks this distinctiveness. The rigorous methodology involving the Web of Science database ensures in-depth interdisciplinary coverage. The survey's meticulous approach in formulating search strategies and selective filtration of relevant articles contributes to its depth and breadth, making it a unique contribution to the field. The significant contributions of this survey paper are listed below:

An overview of the DT definitions, concepts, and architecture in the literature

A Detailed bibliometric study of over 4,220 publications in Digital Twin technology and Smart Cities, including thematic trends analysis like AI and IoT integration.

Examination of datasets, platforms, and performance metrics specific to Digital Twins in urban settings and a critical evaluation of city models.

Applications in Smart Cities: Exploration of Digital Twin technology applications in urban development, encompassing urban planning, energy management, and public health.

Discussion of the challenges in implementing Digital Twin technology in Smart Cities, focusing on data integration, scalability, and security concerns.

Outlining potential research avenues based on current findings, indicating areas for further exploration.

Presentation of practical case studies demonstrating successful Digital Twin integration in urban development.

Summarizing the main findings and implications and a call to action for further research in this evolving field.

The paper's organization follows a clear and structured approach, beginning with Section  1 , an introduction that sets the stage for Digital Twin technology and Smart Cities. It progresses into Section  2 , which provides a detailed bibliometric study, covering objectives, methodology, data collection, and analysis, leading to key findings and implications. Then, Section  3 explores the applications of Digital Twins in Smart Cities. Section  4 discusses some technological aspects of DT. Section  5 presents some examples of datasets and software for developing DT. Section  6 states digital twin performance metrics according to its structure. Section  7 addresses the challenges associated with digital twins. Section  8 introduces some case studies for DT. Section  9 discusses smart city governance in the era of digital twins. Finally, Section  10 summarizes the paper's conclusions and presents future research directions.

2 Bibliometric study on digital twin and smart cities

The primary objective of this research is to perform a bibliometric analysis (Yu and Merritt 2023 ) to acquire a comprehensive understanding of emerging topics, prominent journals, and evolving research trends associated with the application of digital twin technology in smart cities. Additionally, the study aims to shed light on the potential challenges and future research trajectories concerning digital twin technology in the context of smart city development.

2.1 Research methodology

This investigation employed a systematic literature review (SLR) to meticulously explore, assess, and integrate the extant body of knowledge regarding the designated theme, adhering to a rigorously defined protocol (Kyriazopoulou 2015 ). Adopting the SLR methodology is instrumental in delineating the contemporary scholarly landscape of a given topic, thereby uncovering existing research voids and delineating avenues for forthcoming scholarly inquiries (Kitchenham et al. 2009 ). The SLR framework comprises three pivotal phases: planning, execution, and dissemination. The research inquiries were articulated during the planning stage, and criteria for identifying pertinent literature and determining search strategies were established. The execution stage entailed the meticulous gathering and vetting of scholarly works in alignment with the previously established criteria. This phase was initiated with an initial screening of the collected records through their titles and abstracts to ascertain their pertinence to the posed research questions, followed by an in-depth examination of the full-text articles. A bespoke form was devised for the methodical extraction of data, capturing essential information from the chosen articles, such as facets of digital twin components, smart city innovations, and the research lacunae identified therein. Subsequently, the dissemination phase involved the analytical consolidation and synthesis of the compiled literature. The process was underscored by a commitment to transparency and precision, with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) framework guiding the data acquisition methodology (Liberati et al. 2009 ).

2.1.1 Research questions

To delineate the scope of the SLR, the following research questions guided the study:

Q1: What are the components of digital twins in smart city applications?

Q2: What are the existing technologies used in the development of smart city development based on digital twins?

Q3: What are the research gaps and potential areas for future research?

2.1.2 Data collection

This section outlines the steps taken to collect relevant literature for the study. A PRISMA workflow diagram in Fig.  3 illustrates the study's search process. Initial literature searches were conducted in reputable databases such as Web of Science, Direct Science, and Scopus, which were chosen for their extensive coverage of scientific publications and advanced search capabilities. The research strategy applied an advanced search with keywords executed in the Web of Science and Scopus databases with a search string set to ("Digital twin," "virtual twin" or "virtual replica," and "smart city" or "smart cities"), for publications up to September 2023 and set to articles before 2018 were excluded. The period selected for the search is appropriate because there are few publications on digital twins and smart cities before 2018.

PRISMA workflow diagram

2.1.3 Inclusion and exclusion criteria

In this survey, the inclusion and exclusion criteria were meticulously established to ensure a focused and relevant analysis in the fields of Digital Twin technology and Smart Cities. This careful selection was pivotal in delineating the scope of the study.

Inclusion criteria:

Scope of content : Articles must focus on Digital Twin technology and its application within Smart Cities. This includes scholarly articles, conference proceedings, and review articles offering substantial insights into Digital Twin architectures, methodologies for Smart City implementation, technologies employed, and demonstrative case studies or laboratory setups.

Language : Only articles published in English are considered to ensure the clarity and accessibility of the content for our analysis.

Databases : Articles were sourced from the Web of Science, Scopus, and Direct Science databases to ensure a comprehensive and interdisciplinary coverage of the subject matter.

Publication period : Articles published from 2018 to 2023 were included to capture Digital Twin technology's evolution and current state in Smart Cities.

Detail requirements : Articles must present a detailed systematic architecture for a digital twin application and a clear methodology for Smart City implementation. They must also discuss the technologies used and provide demonstrative case studies or laboratory setups.

Exclusion criteria:

Language limitation : Articles published in languages other than English were excluded to maintain consistency and comprehensibility in the analysis.

Irrelevance : Publications unrelated to the direct intersection of Digital Twin technology and Smart Cities, lacking in detailed architecture, clear methodologies, technology discussion, or case studies, were excluded.

Duplication : Duplicated records identified across the databases were removed to ensure the uniqueness and accuracy of the analysis.

Date filter : Articles published before 2018 were excluded to focus the study on more recent developments and applications, reflecting the latest trends and innovations in the field.

The search produced 4,220 records from Web of Science, 382 from Scopus, and 24 from Direct Science. Through a preprocessing step, which involved removing duplicates and applying inclusion and exclusion criteria, 4,507 records were screened. This process yielded 4,073 articles that were deemed eligible for further analysis. The inclusion criteria were specifically targeted at articles that provided detailed systematic architectures for digital twin applications, methodologies for implementing smart cities, descriptions of technologies employed, and demonstrative case studies or laboratory setups.

2.2 Bibliometric study methodology

The methodology of a bibliometric study typically comprises a series of fundamental steps aimed at systematically analyzing scholarly literature within a specific field. These steps include formulating precise research questions to guide the analysis, identifying and selecting appropriate data sources, devising relevant search strategies using carefully chosen keywords, meticulously collecting and preparing the retrieved data, and employing established bibliometric techniques for rigorous data analysis. By adhering to this structured approach, researchers can effectively uncover trends and patterns in scientific publications and citations, thereby gaining valuable insights into the evolving landscape of their area of study (Mora et al. 2019 ). In line with these established practices, this research adopts a systematic approach for collecting, processing, and analyzing academic literature on digital twins within the context of smart cities.

2.3 Data analysis and visualization

This subsection outlines the methodologies and tools implemented to analyze and visualize the bibliometric data. For our study, VOSviewer was selected as the primary tool for managing and interpreting bibliographic data. We utilized network analysis methodologies to generate a range of visual representations. These included co-occurrence analyses, citation and co-citation maps, and keyword co-occurrence maps. Such visualizations were instrumental in uncovering patterns and discerning relationships within the collected dataset.

2.3.1 Publication trends

One of the key indicators in performance analysis is the annual number of publications. This metric serves as an indicator of research productivity. The data collected from 2011 to 2023 reveal a marked increase in publications focused on digital twin technology and smart cities. This surge in research output, demonstrating exponential growth, is depicted in Fig.  4 . This Figure underscores the significance and escalating interest in this interdisciplinary area. Figure  4 illustrates the yearly publication rates concerning digital twins and smart cities. Additionally, Table  1 provides a concise statistical analysis of these findings.

The publications rate of digital twin and smart cities by year from 2011 to 2023

2.3.2 Keyword analysis and research themes

This study's keyword co-occurrence analysis represents a systematic approach to understanding the prevailing keywords associated with digital twin technology and smart cities. The outcomes of this analysis, illustrated in Fig.  5 , reveal a range of predominant research themes and technologies pertinent to the domain of digital twins and smart cities.

Visualization of keyword co-occurrence network

Research themes in digital twin and smart cities:

Theme 1: Integration of AI and big data analytics in digital twins

This theme explores applying advanced deep learning techniques in processing and analyzing digital twin data. Key research areas are identified through terms such as "machine learning," "transfer learning," "simulation," "reinforcement learning," "cloud computing," "AI," "data analysis," "big data," and "forecasting."

Theme 2: Integration of digital twins with IoT

The focus here is IoT technologies, which are central to transmitting and collecting digital twin data. Relevant keywords include "wireless sensor network," "digital devices," "sensors," "5G", "communication," "wireless communications," and "monitoring."

Theme 3: Energy management in digital twins

This theme emphasizes the importance of energy efficiency and sustainability, highlighting keywords such as "energy efficiency," "energy utilization," "sustainability," and "renewable energy."

Theme 4: Security concerns in digital twins

Research in this area deals with the security aspects of digital twins, with keywords like "security," "privacy," "blockchain," and "fault diagnosis".

Theme 5: Cloud computing and digital twins

The final theme investigates the intersection of digital twins with cloud computing technologies, focusing on keywords such as "cloud computing," "edge computing," "fog computing," "blockchain," and "big data analytics."

Predominant technologies in the Digital Twin (DT) domain

Our study analyzed the authors' keywords to ascertain the most prominent technologies within the digital twin sphere. This investigation uses keyword frequency as a metric to identify the key technologies extensively employed in the digital twin (DT) domain. The term 'Internet of Things' (IoT) emerges as the most frequently cited keyword, demonstrated in Fig.  7 . This finding underscores the pivotal role of IoT in the digital twin field, highlighting its extensive research coverage and the ongoing need for in-depth exploration of IoT applications to enhance digital twins' efficacy. Additionally, "AI" and "machine learning" are prominently used to analyze and process large volumes of digital twin data. Other notable technologies such as "cloud computing," "virtual reality," and "digital twin security" have also gained traction. Collectively, these technologies contribute to the efficient storage, visualization, modeling, and security of digital twin data. The data presented in the accompanying table and Fig.  6 substantiate the findings discussed in this subsection.

High-frequency keywords in digital twin research

2.3.3 Analysis of geographical distribution

Examining the geographical distribution in the research and development of digital twin and smart cities technologies offers critical insights into the regional contributions, patterns of collaboration, and prospective areas for advancement. As depicted in Fig.  7 (A), our analysis reveals a broad geographical spread in the field's research activities. We identified key regions contributing significantly to the field by utilizing a citation metric analysis on our dataset, which set a minimum of ten documents and fifty citations per country. China emerges as the leading contributor in terms of citations, followed by the USA, the UK, Italy, and Germany. Furthermore, Fig.  7 (B) corroborates the leadership status of China, the USA, the UK, and Italy in this domain.

A Citations by country in digital twin and smart cities research. B Top 10 publishing countries in digital twin and smart cities research

2.3.4 Analysis of source co-citation

The source co-citation analysis conducted in our study highlights the prominent sources within the domain of digital twins and smart cities. Of 49,275 sources, 433 met the established criterion of a minimum of 50 citations per source. The findings of this analysis are presented in Fig.  8 . The most frequently co-cited journals include IEEE Access, the Journal of Manufacturing Systems, IEEE Transactions on Industrial Informatics, and IEEE Internet of Things. The analysis identified six distinct clusters, each represented by a unique color, as depicted in Fig.  7 (A).

A Co-citation network of sources in digital twin and smart cities research. B Bibliographic coupling network among countries in digital twin and smart cities research

2.3.5 Examination of international collaboration

The observed international collaboration in the digital twin and smart cities sector underscores research's global impact and relevance. Utilizing bibliographic coupling analysis on our dataset, with a set threshold of a minimum of 10 documents and 20 citations per country, 65 out of 103 countries met these criteria. A network visualization visually represents the bibliographic coupling among these countries in Fig.  8 (B).

This analysis collated data on each country's publications, citations, and total link strength. Each node in the figure symbolizes a country whose size reflects its publication count. The visualization reveals that China leads in a collaborative network, boasting approximately 1151 documents and a total link strength of 871,379. Following China are the USA, the UK, and England. Notably, the USA's most extensive collaborations were with China, England, and India, while China's primary collaborations were with the USA, England, and Germany.

The colors in Fig.  8 (B) delineate nine distinct clusters, indicating nations that frequently cite each other's research, suggesting closer collaboration within these groups. This mapping confirms that countries like China, the USA, the UK, Italy, and Germany are at the forefront in advancing research in digital twin and smart cities.

3 Applications of digital twin technology in smart city development

Digital twin technology offers a wide range of applications in smart city development, from optimizing traffic flow and energy usage to improving public safety, Environmental Monitoring and Management, Citizen-Centric Aspects, and Supply Chain Management and Enhancement. By creating virtual replicas of city infrastructure and systems, urban planners and policymakers can visualize potential changes and their impact before implementing them in the physical environment. Furthermore, digital twins can be instrumental in public safety by simulating emergency response scenarios and planning for effective evacuation routes in the event of natural disasters or other crises, as shown in Fig.  9 . As the adoption of digital twins continues to grow, their role in shaping the future of smart cities will become increasingly prominent.

Applications of digital twin technology in smart city development

3.1 Urban planning and management

Urban planning and management encompass the technical and political processes of utilizing land, infrastructure, and buildings within urban areas. This multifaceted domain includes urban design, land use, transportation, zoning, regulation, and environmental planning.

Urban planners and managers increasingly employ digital twin technology to enhance city functions like transportation and sustainability. Digital twins enable more informed decision-making and optimize planning, operations, finance, and strategy. In turn, such systems help reduce carbon emissions and expedite significant projects. Additionally, they enable the simulation of plans before implementation, allowing for the anticipation of potential challenges. The World Economic Forum 2022 recognized the role of digital twins in modeling future sustainable development by integrating digital technology with urban operational systems. This integration facilitates safer, more efficient urban activities. It creates low-carbon, sustainable environments through precise mapping, virtual-real integration, and intelligent feedback of physical and digital urban spaces (Yu and Merritt 2023 ).

Within urban planning and management, digital twins can represent entire cities or specific urban systems, assisting in various ways:

Real-time Monitoring: Integrating sensors and IoT devices with digital twins provides real-time data on urban processes like traffic flow, energy consumption, and air quality.

Simulation and Scenario Testing: Planners can use digital twins to simulate and test different scenarios, assessing the impacts of natural disasters or new transportation systems.

Optimization: Analyzing data from digital twins can identify and address inefficiencies in urban systems.

Public Engagement: Digital twins serve as interactive platforms for public involvement, allowing community members to view proposed changes and provide feedback.

Maintenance and Asset Management: They enable tracking urban infrastructure conditions and predicting maintenance needs.

System Integration: Digital twins facilitate understanding of interdependencies between various urban systems.

Support for Decision-Making: Providing a comprehensive view of the city and its systems, digital twins enhance the decision-making process, ensuring decisions are informed by accurate, up-to-date information.

3.2 Energy management

Energy systems form the backbone of smart cities, ensuring the quality and functionality of these urban environments. This section delves into the application of DTs in energy systems, encompassing transportation systems, power grids, and microgrids.

Digital Twin technology finds varied applications in transportation systems. It supports transportation system infrastructures in several ways, such as monitoring transport systems, traffic forecasting, energy system management, predicting the energy consumption of electric vehicles, IoT-based parking management, analyzing driver behavior, forecasting subway regenerative braking energy, studying pedestrian behavior, controlling health systems, and detecting cyber-physical attacks. DTs can significantly contribute to these areas. For instance, using DTs for transportation system monitoring can reduce maintenance costs. Beyond modeling and planning, DTs facilitate optimal traffic management and provide accurate and extensive traffic and electric vehicle (EV) data, contributing to sustainable development and efficient urban traffic control.

In traffic management, DT technology has been utilized to predict patterns of energy consumption and production (Ketzler et al. 2020 ). Additionally, DTs play a role in IoT-based parking management, improving user services by saving time and reducing parking costs.

Various studies have employed DT technology in diverse contexts. In (Yan et al. 2022 ), authors analyzed real drivers' and pedestrians' behavior using DTs. In (Liu et al. 2020 ; Damjanovic-Behrendt 2018 ), DTs of drivers and vehicles were used in real-time to relay critical information to drivers and vehicles in the physical world. Moreover, in (Crespi et al. 2023 ), the Electric Vehicles (EVs) model employed DTs to monitor the behavior and optimally manage charging programs, using energy consumption parameters and charging capacity and frequency for modeling the virtual twin.

A microgrid is an autonomous energy system characterized by distributed energy resources and interconnected loads. It functions as a manageable entity within the larger grid, enabling it to operate in either island mode or in conjunction with the grid (Ton and Smith 2012 ). The objective of microgrids is to enhance the functionality of energy systems in terms of sustainability, economic viability, efficiency, security, and overall energy management. Key aspects of microgrid performance include reliability, self-sufficiency, security, flexibility, and optimality. Studies on microgrids utilizing the Digital Twin (DT) framework have encompassed areas such as forecasting (Din and Marnerides 2017 ; He et al. 2017 ), management and monitoring (Xu et al. 2019 ; Park et al. 2020 ), fault prediction (Nowocin 2017 ; Goia et al. 2022 ), and security (Huang et al. 2021 ).

The development and implementation of DT-based power grids are instrumental in improving network behavior under various conditions. Network studies employing DT include diverse analyses such as restoration (Biagini et al. 2020 ), reliability (Podvalny and Vasiljev 2021 ), prediction (Park et al. 2020 ), addressing uncertainty (Raqeeb et al. 2022 ), energy hub management (Kuber et al. 2022 ), and ensuring both physical and cyber security. Each of these analyses offers unique insights into network behavior. In reference (Endsley 2016 ), Situation Awareness (SA) is the ability to perceive the elements in a specific environment, understand their properties, and anticipate their future statuses. SA is crucial in augmenting decision-making, especially in complex systems like the Energy Internet of Things (EIoT) (He et al. 2023 ). It provides essential information critical for such systems' operation, enhancing efficiency and effectiveness.

In the application of Digital Twin technology in power systems, several significant challenges emerge:

IT infrastructure limitations: Existing infrastructure often falls short in supporting the data analysis demands of DT environments.

High-performance computing needs: Utilizing high-performance GPUs and cloud services from major providers is essential for adequate support.

Connectivity issues: Software errors and power outages present obstacles in real-time monitoring.

Cybersecurity risks: The extensive data exchange in DT systems heightens vulnerability to cyber-attacks, necessitating secure platforms.

Standardization requirement: The absence of standardized protocols impedes DT development, highlighting the need for unified approaches for model definition, storage, and execution.

The exploration of digital twin applications in energy management reveals several key areas for future development:

Advancements in big models: Addressing challenges in AI, such as limited model generalization and the need for high-quality data, by developing larger, more adaptable models.

Virtual twin structures in power systems: Detailed modeling of power system entities using virtual twins, enabling dynamic visualization and strategy development for urban transformation.

Application of theoretical models: Utilizing chaos and complex system theories (Mir et al. 2022 ) to understand and optimize the nonlinearities in power systems, offering a novel approach to managing system complexities.

3.3 Traffic and mobility management

Traffic and Mobility Management in Smart Cities (Xu et al. 2023 ), enhanced by Digital Twin (DT) technology, represents a significant advancement in urban planning and logistics. DTs enable:

Real-time traffic simulation : Mimicking urban traffic flow to identify and alleviate congestion points.

Public transportation optimization : Analyzing patterns to improve transit routes and schedules.

Pedestrian flow management : Ensuring safer and more efficient pedestrian movement.

Pollution reduction : Aiding in strategies to lower emissions through traffic regulation.

Emergency response enhancement : Assisting in quicker and more efficient routing for emergency services.

Data-driven decision making : Utilizing sensor data for informed traffic management decisions.

Sustainable urban planning : Contributing to long-term urban sustainability goals through efficient mobility solutions.

These applications of DT in traffic and mobility management significantly contribute to creating more livable, efficient, and sustainable urban environments.

3.4 Environmental monitoring and management

Digital twin technology is increasingly integral in urban development, offering real-time insights and solutions for environmental management:

Optimization and prediction: Digital twins, as virtual representations of physical entities, enable process optimization, change monitoring, and future scenario prediction (Wang et al. 2023 ).

Environmental monitoring applications: Usage in water quality monitoring, detecting pollutants, and adapting to changing environmental conditions.

Data integration in smart cities: Interconnection of multiple digital twins, using diverse data sources like temperature and humidity, to forecast environmental conditions (Ivanov et al. 2020 ).

Sensor utilization: Various sensors capture essential environmental data for digital twin construction, including Kinect v2 depth cameras and electronic gloves for manufacturing systems (Nikolakis et al. 2018 ).

Food industry monitoring: Application in monitoring and predicting food quality, employing wireless sensors for environmental factors like humidity and temperature (Defraeye et al. 2019 ).

Agricultural management: Use in agriculture for crop growth monitoring and simulating interventions, aiding in remote farm management (Verdouw et al. 2021 ).

Healthcare applications: Implementation in healthcare for environmental monitoring and mental health management using smartwatch sensors (Bagaria et al. 2019 ).

These diverse applications showcase the role of digital twins in enhancing urban planning, agriculture, healthcare, and more.

3.5 Public health and safety

In the development of Smart Cities, Digital Twin Technology plays a crucial role in enhancing public health and safety (Erol et al. 2020 ). It offers a dynamic and integrated approach to managing complex urban health challenges through simulation and analysis. This technology's applications include:

Disease outbreak prediction and management : Leveraging real-time data to simulate disease spread and plan responses.

Emergency preparedness : Using simulations for natural disasters or public safety incidents to enhance response strategies.

Resource optimization in healthcare : Improving the allocation of healthcare resources like hospital beds and emergency services.

Environmental health monitoring : Tracking and analyzing environmental factors that impact public health, such as pollution levels.

Public safety and incident response : Simulating various scenarios to optimize law enforcement and emergency services.

These applications demonstrate the transformative impact of Digital Twin Technology in public health and safety within Smart Cities.

3.6 Citizen-centric aspects

Technological advancements have focused on urban development and infrastructure management, employing physical sensors such as the Internet of Things (IoT) and satellites (Borrmann et al. 2018 ). However, not all city digital twin implementations have citizen engagement in mind. A citizen-centric digital twin (CCDT) approach views citizens as integral components of a data-driven city, with human sensors playing a key role in addressing city-scale challenges (Saeed et al. 2022 ). This approach distinguishes itself from traditional digital twin frameworks by prioritizing citizens as the central element and integrating technologies like processing, data acquisition, and visualization to enhance citizen involvement in infrastructure governance.

Developing a CCDT requires the execution of numerous processes and technologies. One such technology involves sensors like Volunteered Geographic Information (VGI) (White et al. 2021 ), which transfer data from the actual city to its digital twin, followed by analysis using various analytical tools. Managing data from diverse sources at a city scale presents challenges in scalability, reliability, and the performance of real-time analytics and modeling (Langenheim et al. 2022 ).

The study by Abdeen et al. (Abdeen et al. 2023 ) indicates a scarcity of publications in this field over the past five years. However, a rising trend in CDT interest post-2017 was noted, with publications doubling by the end of 2022 compared to 2019. Research works (Ford and Wolf 2020 ; Fan et al. 2021 ) discuss the application of digital technologies in catastrophic situations and emergency responses. The capabilities of intelligent digital twins in various application fields have been examined (Shahat 2021; Deren et al. 2021 ), with the latter focusing on hazards like epidemic services, traffic control, and flood monitoring. (Shahat et al. 2021 ) concentrated on data simulations, fusion, administration, and collaboration. (Charitonidou 2022 ) addressed citizen participation in decision-making, highlighting that limited variables and processes and overlooking social aspects of urban contexts can render citizen input integration ineffective.

In the literature, various data acquisition mechanisms are employed to support CCDTs. One prominent method is the use of open-source data platforms (OSDP), providing spatiotemporal performance data relevant to CCDT applications like disaster management (Ghaith et al. 2022 ) and public services monitoring (Diakite et al. 2022 ). However, the effectiveness of CCDTs can be compromised if data is unreliable. Another mechanism is crowdsourcing, which generates large quantities of data and is particularly useful when remote or IoT sensors are unavailable (Trusov and Limonova 2020 ). Nevertheless, citizens' data errors or bogus inputs can affect CCDT effectiveness (Trusov and Limonova 2020 ). Visionary concepts for disaster city digital twins with extensive data (images, text, geo maps) have been proposed to enrich CCDT content (Fan et al. 2021 ).

Remote sensors are effective in modeling 3D city aspects of CCDTs and for large-scale urban monitoring (Fan and Mostafavi 2019 ; Fan et al. 2020 ). Geospatial platforms storing and managing data from individual vehicles or pedestrians have been proposed (Lee et al. 2022 ), though data accessibility to all stakeholders remains challenging for CCDT integration. Furthermore, IoT sensors (Nochta et al. 2020 ), deployed in large numbers and integrated effectively, facilitate urban data monitoring but require advanced communication infrastructure.

Advanced AI algorithms also play a crucial role in CCDTs, enhancing citizen engagement. Genetic algorithms (Fan et al. 2021 ) have been used to study the range of disruptions during hazard events, while Convolutional Neural Networks (CNN) (Pang et al. 2021 ) and Burst detection algorithms (Fan et al. 2021 ) help analyze crowdsourced data and social media frequencies providing insights into citizen perspectives and infrastructure governance through CCDTs.

3.7 Supply chain management and enhancement

The supply chain, encompassing the entire spectrum from raw material sourcing to the distribution of finished products, has seen a transformative integration of Digital Twin technology in recent years. Digital Twins, as virtual models of physical assets, offer real-time monitoring, analysis, and optimization across all facets of the supply chain, ranging from procurement to distribution (Tao et al. 2017 ). According to van der Valk et al. (van der Valk et al. 2022 ), these digital replicas enable two-way data exchange between the digital and physical worlds, providing professionals with exceptional visibility and traceability. This level of insight facilitates the identification of complex behavioral patterns and proactive problem detection, which is crucial for maintaining operational continuity.

As Gerlach et al. (Gerlach et al. 2021 ) highlight, Digital Twins are instrumental in offering real-time inventory insights, enabling the simulation of various scenarios, and assisting in planning and forecasting. These capabilities can result in significant cost reductions and process efficiency improvements. The study by Srai et al. (Srai and Settanni 2019 ) explores the optimization opportunities that Digital Twins offer in areas such as transportation resource management, demand–supply analysis, customer service improvement, and revenue enhancement. They also emphasize the role of technology in identifying and addressing inefficiencies.

The influence of Digital Twins in improving stock availability, a key aspect of manufacturing operations, is underscored by Abouzid et al. (Abouzid and Saidi 2023 ). Furthermore, (Lugaresi et al. 2023 ) introduces the concept of "technological labelers" like IoT devices, cloud computing, and advanced analytics, which are crucial in developing a comprehensive digital twin of a company's value chain. IoT integration, in particular, is noted for significantly enhancing supply chain efficiency by providing real-time data and contextual insights. In addition, distorted demand signals can result in various supply chain challenges, which can be effectively addressed using Digital Twin technology (Abouzid and Saidi 2023 ).

In conclusion, the discourse delves into the exploration and development of digital twins within automated manufacturing systems, showcasing the expansive potential of this technology in modernizing and streamlining supply chain management processes (Abideen et al. 2021 ).

4 Technological aspects of digital twin

This section explores the fundamental components and emerging advancements in digital twin technology, covering various technological aspects essential for understanding Digital Twins across disciplines. Each component offers unique insights into critical subjects, including distinctions between Digital Twins and Building Information Modeling (BIM) and the development of Cognitive Twins. By studying frameworks like the Five-Layer Architecture and advancements such as Cloud and Edge Computing integration, this section aims to reveal the technological foundations driving the evolution of Digital Twins. Readers will gain a deeper understanding of the technological breakthroughs shaping the future of Digital Twins and their applications through this comprehensive examination.

4.1 Distinction between digital twins and Building Information Modeling (BIM)

In construction technology, the emerging potential of digital twins and the rapid advancement of smart technologies has garnered significant interest. Although 'digital twin' is a relatively new term in the construction research literature, it is often conflated with Building Information Modeling (BIM), leading to some conceptual ambiguity. It is imperative to clarify the differences between these two concepts.

Building Information Modeling (BIM) is a digital representation of a building or structure's physical and functional characteristics. It is a tool architects, engineers, and construction professionals utilize to create detailed digital models of buildings, encompassing all systems and components, including architectural, plumbing, electrical, and HVAC systems. BIM enables the creation of accurate construction plans, virtual walkthroughs, and performance testing under various scenarios. (Wang and Meng 2021 ) defined BIM as a method that integrates geometric and non-geometric data. The 3D model, often called the BIM model and realized through object-oriented software, is a critical component of BIM (Cerovsek 2011 ). However, BIM primarily manages static data and requires external technologies to update models with real-time data (White 2021). In construction projects and asset management, a vast amount of non-geometric data is essential for informed decision-making but is often underutilized (Khudhair et al. 2021 ). BIM models have limited capacity to handle large volumes of dynamic and multifaceted data, necessitating advanced storage and processing technologies. These limitations can lead to data underutilization, inefficient decision-making, and financial implications. The advent of digital twin technology offers a solution to overcome these constraints inherent in BIM.

Digital twins and BIM represent two distinct technological applications in the construction sector, differentiated by their functions. BIM is most effective in the design and construction phases, while digital twins excel in building maintenance and operations. A digital twin system involves data linkage that transfers information between the physical asset and its virtual counterpart. This indicates that a BIM (Building Information Modeling) model is the initial step toward developing a digital twin in the construction industry. Digital twin technology integrates the BIM model with the physical world, enabling bidirectional data exchange. This connection allows for the real-time updating of the BIM model, enhancing asset implementation and management decision-making. The synergy between BIM and Digital Twin technology has the potential to revolutionize the construction industry. By combining the detailed architectural and structural information provided by BIM with the real-time operational data and analysis capabilities of Digital Twin technology, construction professionals can create comprehensive, accurate digital models of buildings. These models can then continuously monitor and optimize building performance in real-time.

4.2 Framework of the five-layer architecture in digital twins

A digital twin, essentially a digital representation of a physical entity, process, or person contextualized in a virtual environment, is a pivotal tool for organizations to simulate real-world scenarios and outcomes, thus enhancing decision-making capabilities (Moosavi et al. 2021 ). As shown in Fig.  10 , the architecture of digital twins is typically structured into five principal layers, as outlined in (Jones et al. 2020 ):

Five-layer architecture in digital twins

Physical layer : This foundational layer comprises the actual physical objects or entities. It utilizes sensor technology for data acquisition and can receive commands from the virtual layer. This layer provides real-time data feedback to the digital twin model.

Data sensing layer : Responsible for collecting diverse information types, this layer employs various sensors to monitor the system's status and operational process in detail. The data heterogeneity and variety stem from diverse data generation sources, such as IoT sensors, information systems, and wearable devices.

Data transmission layer : As a crucial link, this layer ensures data transmission between the physical and virtual layers. It leverages communication integration protocols and interactive security technologies to facilitate this transfer (Lohtander et al. 2018 ).

Virtual layer : In this layer, the components of the real world are digitally reconstructed. It builds a collection of digital twins using data transmitted from the physical layer, enhanced with historical or integrated network data. This layer dynamically tunes itself based on the real-time data from the physical layer and can be influenced by modifications made in the application layer.

Application layer : This layer visualizes the data and simulations derived from the virtual layer, presenting a graphical model that staff can easily interpret. Modifications in the physical or virtual layer parameters can lead to simulation changes. These can then be revised and optimized based on the observed or extrapolated results.

Each layer in this five-tier architecture plays a distinct yet interconnected role, collectively enabling the digital twin to function as a comprehensive, dynamic system for analyzing, simulating, and enhancing real-world processes and entities.

4.3 Integration of cloud and edge computing in digital twin environments

Cloud computing represents a large-scale computational approach that leverages the Internet to facilitate sharing computing, storage, and other resources, accessible anytime and anywhere on demand. In contrast, edge computing is a novel computational model that processes a portion of data using distributed computing, storage, and network resources between data sources and cloud computing centers.

Edge computing is increasingly recognized for its potential to enhance privacy, reduce latency, conserve energy and costs, and boost reliability. It is particularly well-suited for Digital Twin (DT) scenarios that demand low latency, high bandwidth, high reliability, and stringent privacy measures. In DT-assisted edge computing setups, the framework includes user devices, edge servers, resource devices, and the DT itself. User equipment initiates task requests to the edge server, which then allocates computing resources to the task, with the DT deployed within the edge server. Reviewed literature demonstrates the application of Cloud and Edge technologies in various contexts. Cloud storage is universally employed in these studies. Earlier research also utilized cloud computing for user interfaces, with cloud-rendered 3D models, as indicated in (Xu et al. 2021 ), or through GUIs accessible via web applications as in (Urbina Coronado 2018). While initial studies relegated intensive data processing and analytics to cloud computing due to its superior resource access, recent advancements in edge device capabilities have led to the implementation of edge computing, employing techniques for heavy data analysis or machine learning (Cathey et al. 2021 ; Lu et al. 2021a ; Zhang et al. 2022 ).

Recent digital twin research, such as (Alam and El Saddik 2017 ), employs edge computing, describing a framework where each device is represented as a cloud-based digital twin. This hierarchical architecture involves higher-level digital twins composed of simpler units in a master/slave relationship, enhancing the communicability of traditional cyber-physical systems with cloud servers' advanced computational and storage capacities. Focus on edge-based architectures is evident in (Dong 2019; Lu et al. 2021a ), with research by Dong (Dong 2019) on enhancing energy efficiency in 5G services through deep neural networks and Lu (Lu et al. 2020 ) exploring the use of digital twins in-network replication and machine learning via federated learning.

In industry, studies such as (Lu et al. 2021b ; Zhang et al. 2022 ) concentrate on Smart Vehicles, driven by the rise in edge computing power. Other research, including (Liu et al. 2019 ; Martinez-Velazquez et al. 2019 ), investigates the application of digital twins in healthcare, aiming to provide high-quality, real-time care to senior citizens. Most other studies, such as (Xu et al. 2021 ; Urbina Coronado et al. 2018 ; Hu et al. 2018 ; Bellavista et al. 2021 ), are categorized under Smart Manufacturing, focusing on industrial productivity improvements. Cloud-based digital twins play a crucial role in optimizing IoT device energy consumption and operational efficiency (Li et al. 2020 ), detecting and preventing potential system failures (Cathey et al. 2021 ), and ensuring data privacy and integrity (Wen et al. 2020 ). Thus, cloud computing and IoT emerge as complementary technologies, synergistically advancing the development of smart, interconnected systems.

Research in the Oil and Gas industry reflects a systematic adoption of digital twins and cloud/edge computing. For instance, (Pivano et al. 2019 ) discusses offloading simulations and data analysis to public cloud servers to access greater computational resources and avoid complex local IT infrastructures. Tygesen et al. (Tygesen et al. 2018 ) highlight the role of high-performance cloud computing in wave load modeling, which is essential for maintaining offshore platform integrity. (ASME 2018 ) describes the use of cloud data lakes for data verification and physical model feeding. At the same time, a microservices-based approach has been presented for designing and implementing digital twins using open-source tools (Zborowski 2018 ).

4.4 Implementation of augmented reality in digital twin technology

Augmented Reality (AR) is a technology that merges the real with the virtual, facilitating real-time interaction and 3D registration (Damjanovic-Behrendt and Behrendt 2019 ). It enhances user experience by superimposing graphics, video streams, or holograms onto the physical world (Yin et al. 2023 ). It is supported by various devices such as AR head-mounted displays (HMD), tablets, head-up displays (HUD), projectors, VR HMD with cameras, and 2D screen augmentations.

AR's enhancements are primarily derived from its visualization, interaction, 3D registration, and information collection capabilities as a unified device (Billinghurst et al. 2015 ). AR contributes to Digital Twin (DT) technology in several dimensions. In the virtual twin dimension, AR provides visualization of non-registered geometry, data, workflows, and basic status monitoring and alerting for operators. It also allows users to update DT information, for example, by scanning barcodes or adding annotations. However, the full potential of AR in augmenting DTs remains underexploited. In the hybrid twin dimension, AR enables multi-modal interactions and on-site registered visualization, with a need for further exploration and utilization in cyber-physical interaction functions. In the cognitive twin dimension, AR-assisted DT, bolstered by edge-cloud computing systems, is poised to play a more significant role in areas like visual programming, human–robot collaboration (HRC), product design, and human ergonomics, marking promising future directions for AR-assisted DT.

Applications of AR-assisted DT span a wide range of physical scenarios, encompassing the entire product lifecycle, including the management of production facilities and services. The production process and service phases include design, production, distribution, maintenance, and end-of-life stages, as illustrated in Fig.  11 .

Applications of AR-Assisted digital twins in engineering lifecycle, including design, production, distribution, maintenance, and end-of-life stages

The systematic design process involves prototyping, pilot runs, and testing operations. Real-time data from product usage, collected by sensors, informs smart product service design or redesign, integrating DT in mapping virtual and physical objects (Praschl and Krauss 2022 ). Research in AR device utilization for design falls into three categories: product design (Zheng et al. 2018 ), service design, and system design. Service design, a creative and user-centric process for enhancing or creating new services (Chang et al. 2020 ), is supported by several studies focusing on operation training (Blomkvist et al. 2023 ), driving and flight guidance (Moya et al 2020b ), smart environments (Vidal-Balea et al. 2021 ), smart cities (Lacoche and Villain 2022 ; Ssin et al. 2021a ), and smart wetlands (Ssin et al. 2021b ), aiming to deliver user-centered services that cater to the needs of users and stakeholders.

System design entails developing architecture, components, and core algorithms for AR-assisted DT scenarios. Moya et al. (Aheleroff et al. 2020 ; Moya et al. 2020a ) introduced two self-learning DT systems with screen augmentation for fluid behavior prediction and beam load analysis.

The production process includes goods fabrication or service provision, subdivided into process planning and scheduling (Wiegand et al. 2018 ), monitoring and control (Lemos et al. 2022 ), assembly (Kritzler et al. 2017 ), and robotics-related works (Židek et al. 2021 ). Real-time machine status monitoring and interactive control are prevalent in research, as demonstrated by Paripooranan et al. (Paripooranan et al. 2020 ), who developed an AR-enabled 3D printer DT for alerting abnormal statuses.

In distribution, warehouse management utilizes AR and DT, as shown by Petković et al. (Petković et al. 2019 ) in their use of a warehouse system DT (comprising the warehouse, automated guided vehicles (AGV), and operators with AR HMD) to test a human intention estimation algorithm.

Maintenance work encompasses various strategies and can be categorized into reactive, preventive, and predictive maintenance (Petković et al. 2019 ), adopting different approaches within the AR-assisted DT framework.

4.5 Hybrid twins in mixed reality applications

Mixed Reality (MR) applications offer an interactive experience that blends real and virtual environments, akin to Augmented Reality (AR) (Damjanovic-Behrendt and Behrendt 2019 ). Additionally, the term Extended Reality (XR) encompasses Virtual Reality (VR), AR, and MR and has been included in the research scope. The enhancements brought about by AR are examined across three distinct dimensions of the digital twin: the virtual twin, hybrid twin, and cognitive twin, as depicted in Fig.  12 .

Layered framework for digital twin classification based on augmented reality devices' perceptual capabilities

The virtual twin dimension encompasses data transmission from physical to virtual realms, non-registered visualization, and essential status monitoring and alerting functions based on sensor data. When enhanced by Augmented Reality (AR) devices' perceptual capabilities, this dimension can improve the data transmission process from the physical to the virtual space and suitably update Digital Twin (DT) information. Beyond IoT sensor data, on-site information such as barcodes and workspace details can also be gathered through AR applications, exemplified in warehouse management (Xia et al. 2022 ).

Reference (John Samuel et al. 2022 ) discusses the concept of hybridization in DTs, focusing on refining DT accuracy through self-adaptation and data-driven estimation techniques. This approach integrates physics-based model predictions with process measurements, creating a hybrid digital twin (HT) that facilitates the soft-sensing of otherwise hard-to-predict data.

The hybrid twin dimension emphasizes analysis and feedback from the virtual to the physical world, such as context information-related analysis, visual registration, multi-modal interaction and control, and the functionalities based on these aspects. Traditional DTs manage real-time data analysis, including simulation, prediction, diagnosis, and optimization, feeding back the analysis outcomes from the virtual to the physical world. AR-assisted DTs enhance this analysis with on-site data, adding capabilities like object localization, scene understanding, and cyber-physical interaction computation. For instance, in human–robot collaboration (HRC) assembly (Johansen et al. 2023 ), the hybrid twin dimension offers immersive visual registration beyond traditional 2D interfaces, displaying geometry and key data overlaid on the physical entity in the correct position. In contexts such as assembly (Liu et al. 2022 ; Zhao and Sun 2020 ), maintenance (Meier et al. 2021 ; Li et al. 2021 ; Rabah et al. 2018 ), and manual or semi-automated tasks (Koteleva et al. 2021 ; Rebmann et al. 2020 ; Mandl et al. 2017 ), operators can reference on-site instructions and guidance to work more efficiently. Additionally, geometry overlay for inspection (Catalano et al. 2022 ; Xie et al. 2020 ) or motion preview (Kim and Olsen 2021 ) aids operators in verifying the shape or movement of physical entities against planned outcomes. Users can also add geometry-linked or position-related annotations through AR.

Akroyd et al. (Akroyd et al. 2022 ) introduced the concept of the Universal Digital Twin, a digital twin that leverages a dynamic knowledge graph to enable cross-domain interoperability for DTs.

4.6 Development of cognitive twins in digital twin technology

Cognitive twins represent an advanced form of digital twins endowed with high-level cognitive capabilities encompassing machine and human intelligence. These cognitive twins are designed to address complex and unpredictable situations using enhanced computational power dynamically. Augmented Reality (AR) significantly contributes to the development of cognitive twins as it can function as a wearable computational unit within the edge-cloud architecture (Li et al 2022 ). HoloLens 2, a widely-used AR device, notably possesses substantial computing power (1 T FLOP) compared to wearable devices like sensors. This capability allows training models on high-power devices and their subsequent deployment on HoloLens 2, highlighting one of AR's key benefits to digital twins.

Cognitive Digital Twins (CDTs), originating from the domains of Industry 4.0 and Smart Cities, are recognized for their ability to support autonomous activities (Um et al. 2018 ; Liu et al. 2023 ; Zheng et al. 2021 ). Semantic technologies, including ontology and Knowledge Graph (KG), are vital in interlinking digital twins in virtual spaces. These technologies eliminate ambiguity across heterogeneous systems, thus enhancing digital interoperability and enabling cooperative decision-making (Rožanec et al. 2022 ). As defined by (Pan et al. 2021 ), ontology involves a set of formal and explicit vocabularies characterized by shareability and reusability, describing domain-specific knowledge, entities' attributes, and their interrelationships. While early research primarily focused on utilizing ontology for data modeling and sharing (Rožanec et al. 2022 ), recent studies emphasize that integrating semantics with digital twin technologies can advance the capability and interoperability of CDTs in autonomous and cooperative decision-making (Zheng et al. 2021 ).

The knowledge graph has become increasingly important in developing and managing CDTs because it can delineate relationships between real-world entities or link data (Liu et al. 2023 ). For instance, recent research has explored using knowledge graphs and digital twins in managing assets and tasks in smart manufacturing systems (Guarino et al. 2009 ) and underwater ship inspections (Zheng et al. 2023 ). Some studies have concentrated on methodologies that leverage knowledge graphs to create semantic data models for shaping digital twins (Waszak et al. 2022 ).

Furthermore, the evolving flexibility and customization in futuristic smart manufacturing are closely linked with human intelligence. For example, in human–robot collaboration (HRC) tasks aimed at improving human ergonomics (Steinmetz et al. 2022 ), operators can adjust robot poses through gesture-based interactions with the robot's digital twin. After receiving instructions from human operators, the robot digital twin learns to perform better and meet human needs. Additionally, in the timber prefabrication process (Dimitropoulos et al. 2021 ), AR provides effective interaction methods to enhance mutual understanding between operators and collaborative robots, ultimately facilitating harmonious task sharing.

4.7 Classification of digital twins by scale

Digital twins can be categorized into various types based on their scale and comprehensiveness, including component, asset, system, and process twins (Amtsberg et al. 2021 ).

Component twins : This approach suits large, complex digital twins. The adaptation and uncertainty quantification of the model in such applications can be framed as a Bayesian state estimation problem. Here, data from the physical world is used to infer which models from a model library best represent the digital twins. This approach strategically selects specific components for replication in the digital twins to avoid data redundancy and reduce costs. Microsoft has developed the Azure Digital Twins (ADT) platform (Cinar et al. 2020 ), facilitating model creation and offering a graph API for querying and interacting with these digital twins. The ADT platform enables users to visualize and examine the relationships among components, such as creating 3-D digital twins of a factory with a user-friendly interface. This interface allows operators to monitor the state of each machine. A notable challenge in this scenario involves loading each 3-D object instance into the scene. Repeated loading of the same object in different locations can lead to inefficiencies.

To address this, future developments in component twins could involve a system where a single instance of a 3-D object is streamed, loaded into memory, and rendered multiple times as needed. This approach would optimize the handling of 3-D objects in digital twin environments, enhancing efficiency and reducing the computational load.

Asset twins : This methodology focuses on creating data-driven digital twins using a library of physics-based reduced-order models. When a single model library is shared among numerous assets, this approach can effectively scale to applications requiring a substantial number of digital twins (Krzyczkowski 2019 ). Asset twins involve an estimation process wherein online sensor data from a physical asset determines which models from the library should be integrated into the digital twin. Future advancements in asset twins should enhance the robustness of model selection, particularly in the context of corrupted data. Implementing mechanisms to improve robustness and incorporating various damage models to detect and classify actual asset damage is also essential. GE Healthcare (Kapteyn et al. 2020 ) has noted the application of asset twins in healthcare, addressing challenges such as staffing model design and surgical block schedule optimization.

System twin : Operating at a higher level, system twins amalgamate different assets to form a complete functional system, such as a vehicle's brake system (Aghdam et al. 2021 ). These twins offer insights into asset interactions, thereby augmenting overall performance.

Process twin : Process twins utilize high-performance computing to optimize equipment and manufacturing processes. This is achieved by integrating multidimensional process knowledge models (Aghdam et al. 2021 ). Manufacturers can attain unparalleled efficiency and deeper insights by combining production processes with economic considerations.

Application : A digital twin system integrating Virtual Reality (VR) and Artificial Intelligence (AI) technologies has been developed to monitor and analyze welder behavior. This system exemplifies the practical application of digital twin technology in understanding and improving specific work processes.

5 Datasets, data models, and software for developing digital twins

The transformation of physical assets into digital twins involves an in-depth asset data collection process, which is then utilized to form an exact digital counterpart. This procedure is essential for asset management and predictive maintenance. There are variant data models and datasets used to underpin the digital twin initiative and significantly enhance the effectiveness and capabilities of digital twin implementations while reducing development efforts and optimizing the total cost of ownership. Many software applications have recently been used to create and manage digital twins. This section presents samples of Data models, Datasets, and software applications.

5.1 Smart city data models and datasets

To illustrate the potential of digital twins in smart cities, let us consider examples of digital twin data models and datasets that provide valuable insights for urban planning and management. Digital twin data can be applied in both tangible and virtual realms. These data are pivotal for asset monitoring, operational optimization, and safety enhancement in physical settings. On the other hand, virtual landscapes enable realistic simulations, training endeavors, and strategic planning. This dual use of digital twins highlights their adaptability, effectively bridging the real and digital domains.

One of the cornerstones of DT design and development is modeling data. Data originate from heterogeneous sources, use various protocols, and include their own data attributes, attribute types, and relationships. In order to ensure interoperability, it is necessary not only to standardize the communication between DT components but also to standardize the data format that flows through these components.

3D city modeling transcends mere data acquisition and processing, extending into data management, storage, and exchange. Consequently, open and standardized data models and exchange formats are essential for 3D city modeling. CityGML and its streamlined counterpart, CityJSON (Ledoux et al. 2019 ), are the most established data formats for 3D city models. These formats facilitate representations ranging from basic to richly detailed, depending on the required level of detail (LoD). The building model is depicted in five levels of detail, from LOD0 to LOD4, with higher LoDs offering more detail and accuracy. The aim is to manage the complexity of 3D models effectively.

In their study, the authors (Lei et al. 2022 ) assess 40 authoritative 3D city models that have emerged since 2013. This evaluation yields both quantitative and qualitative insights. The framework developed offers a thorough and structured comprehension of the landscape of semantic 3D geospatial data while also serving as an evaluated compilation of open 3D city models.

In (Ledoux et al. 2019 ), digital twin (DT) initiatives in cities are classified based on the nature of their digital replicas (static or dynamic, i.e., incorporating sensor or IoT data) and the extent of data integration (the data connection between the physical and digital worlds). Various static datasets utilize digital model integration, including Helsinki 3D + , Espoo, Vienna, Zurich 3-4D, and Amsterdam3D. Meanwhile, dynamic datasets such as Digital Twin Munich, Rennes 3D, Virtual Gothenburg, and Sofia-Bulgaria employ digital shadow integration. Furthermore, dynamic datasets like DUET, Fishermans, and Virtual Singapore implement digital twin integration. It can be inferred that most initiatives are digital shadows, given that data connections from the real world to the digital copy are automated. At the same time, the reverse typically involves manual processes (human interventions adapting the physical world). This bidirectional connection warrants further exploration.

Many research projects and similar initiatives mainly focus on collecting and providing IoT data generated from smart cities. For example, the ODAA platform ( 2016 ) Footnote 1 provides open access to data collected from the City of Aarhus using IoT infrastructure deployed within the city. The datasets within the ODAA are categorized across various applications, including energy, population and society, transport, education, and more. Moreover, San Francisco Open Data ( 2024 ) Footnote 2 and the City of Chicago Data Portal ( 2024 ) Footnote 3 provide a centralized collection of relevant smart city datasets that are publicly accessible.

For example, the NYC Open Data Initiative has already leveraged digital twin technology to improve urban planning and citizen engagement. By providing access to a wide range of open data, including information on infrastructure, public services, and environmental factors, the initiative has empowered citizens to actively participate in shaping the city's future.

5.2 Software for digital twin creation and management

Numerous digital twin software applications are available for creating and managing digital twins in buildings, cities, and urban systems. Some notable examples include:

Autodesk revit (Autodesk 2019 ): This software is extensively used for Building Information Modeling (BIM) and is acclaimed for its comprehensive design, documentation, and collaboration tools. It enables architects, engineers, and construction professionals to create detailed 3D models and provides extensive data for informed decision-making throughout a building's lifecycle.

Esri cityengine (Badwi et al. 2022 ): CityEngine is a robust software tool for crafting 3D city models. It is utilized by urban planners and designers to generate detailed and lifelike representations of cities, offering capabilities for cityscape generation, urban environment modeling, and simulation of various urban scenarios. It also integrates with GIS data to enhance city models with geographic information and analysis.

Bentley systems openbuildings designer (Mainisa et al. 2023 ): This BIM software provides advanced building design and construction modeling tools. Architects, engineers, and construction professionals use it for detailed 3D modeling, structural analysis, and effective collaboration throughout the building lifecycle.

Unity reflect (Nämerforslund 2022 ): Unity Reflect is a platform that creates interactive and immersive experiences with digital twins. It supports real-time, high-fidelity 3D modeling for virtual and augmented reality environments, enhancing visualization, interaction, and decision-making processes.

Siemens city performance tool (Al-Obaidy et al. 2022 ): Specifically tailored for urban planning and management, this tool offers a comprehensive platform for analyzing and optimizing the performance of urban systems.

iLens from knowledge lens : This leading Industrial IoT solution addresses Industry 4.0 needs with capabilities in Interface Connectivity, Edge Computing, Monitoring and Control, and Predictive Analytics. iLens is powering diverse industries globally, including Automation, Manufacturing, Energy, and Utilities.

Iotics : Iotics' innovative digital twin technology enables seamless communication across an entire digital ecosystem. It bridges gaps between various entities, from sensors to power stations and individual trains to entire airplane networks, transcending organizational boundaries and differing data languages while maintaining security.

Kavida.ai : This supply chain digital twin platform assists enterprises in making intelligent resiliency decisions. It builds supply chain digital twins using artificial intelligence to help enterprises prevent and mitigate disruptions in real time or before they occur.

MODS reality : This cloud-based application hosts a digital twin of a facility in a point cloud environment, enhancing engineering and streamlining scheduling and work execution management for maintenance and minor modifications, thereby maximizing performance and profitability.

Twinzo : As a mobile-first live digital twin platform focused on operational excellence, twinzo visualizes and reconstructs live data in 3D, offering novel ways to analyze and consume information. It helps customers save significant operational costs and increase production output.

VEERUM's digital twin : This application is a leading visualization and analytics tool that combines CAD, geospatial, document management, IoT, and operational systems. It delivers considerable cost and time savings in operations, maintenance, reliability, and complex capital construction projects.

WillowTwinTM : Revolutionizing the built world, WillowTwinTM is a pioneering software platform for real estate and infrastructure assets. It provides a central hub for all asset data, turning siloed datasets into a virtual replica of the built form. The platform enables proactive, data-driven decision-making in real-time to reduce costs, increase profits, and manage risks.

6 Digital twin performance metrics

Extensive research has been conducted on digital twins (DTs) and their applications, yet a standard method for assessing DT performance remains elusive. Establishing a method for evaluating the performance of DTs is essential for enhancing or monitoring processes and systems within a business context. Such a method could guide researchers and practitioners in developing more effective digital twins (Psarommatis and May 2022 ).

There have been limited studies focusing on specific methodologies for assessing DT performance. Chen et al. (Chen et al. 2021 ) proposed a DT maturity model for managing industrial assets based on Gemini principles, facilitating quantitative evaluation of DT flexibility and implementation levels. Chakraborty et al. (Chakraborty and Adhikari 2021 ) assessed DT performance in a multi-time scale dynamical system using an efficient framework that leverages expectation maximization and a sequential Monte Carlo sampler for developing machine learning-based DTs. Shangguan et al. (Shangguan et al. 2020 ) evaluated DT performance for fault diagnosis using a predefined threshold technique, focusing on accuracy (ACC), specificity (SPE), and sensitivity. Psarommatis et al. (Psarommatis and May 2022 ) introduced a systematic approach for measuring DT performance and flexibility, quantifying it based on four key performance indicators (KPIs). Additionally, they introduced DTflex as a new KPI to evaluate the flexibility of digital twins.

6.1 Performance metrics categories

Although there are no well-established methods or Key Performance Indicators (KPIs) in the field for thoroughly assessing the performance of Digital Twins (DT), this study suggests classifying performance metrics according to three essential elements: software, hardware, and data management middleware. This paradigm makes it possible to evaluate the system's efficacy in detail. A thorough analysis of the body of prior research and industry norms guided the choice of these indicators. We aimed to find measures that captured the essential elements of DT performance by combining knowledge from several sources.

The proposed metrics ensure an adequate evaluation by focusing on DT performance characteristics within each component. For example, metrics about hardware components evaluate attributes like scalability, communication dependability, and sensor precision. These metrics were selected to represent the fundamental hardware performance features essential to DT's operation. Similarly, metrics related to middleware for data management emphasize security, scalability, and efficiency, highlighting middleware's vital role in integrating and controlling data streams. Finally, software component metrics highlight the significance of strong software functions for DT performance by addressing factors such as model integrity, simulation accuracy, and user interface responsiveness. Each metric recommended in this section is supported by its relevance to real-world DT implementations and alignment with broader business or operational objectives. These measures help stakeholders make well-informed decisions by offering practical insights about DT performance. Including these measures also attempts to create a standard framework for assessing DT performance in various applications and domains.

6.1.1 Hardware components

Sensor accuracy: Precision and reliability of physical sensors.

Communication reliability: Efficiency of data transmission between sensors and the digital counterpart.

Hardware scalability: Ability to expand hardware components with increasing data volumes.

Latency in data acquisition: Time taken to acquire and transmit sensor data.

Hardware failure rate: Frequency and severity of failures in sensors or actuators.

6.1.2 Data management middleware

Data integration efficiency: Ease of integrating data from various sources into the DT.

Middleware latency: Time taken for middleware processes to complete tasks.

Data accuracy and consistency: Precision and consistency in data storage and management by middleware.

Scalability of middleware: Ability to handle increasing data volumes without performance degradation.

Data security protocols: Effectiveness of security protocols in protecting data during storage and transit.

6.1.3 Software components

Model fidelity: Accuracy and completeness of the digital model representing the entity.

Simulation accuracy: Precision of simulations compared to real-world scenarios.

Quality of visualization: Clarity and detail of visual representations in the user interface.

User interface responsiveness: Speed and responsiveness of the software interface to user actions.

IoT device integration: Compatibility and integration with various IoT devices.

Scalability of software: Capacity to handle increasing computational loads and data processing demands.

Software security: Protections against cyber threats and unauthorized access.

Interactivity and control: Responsiveness of software to user inputs and control commands.

Updating and maintenance efficiency: Ease of updating and maintaining software components.

Effectiveness of decision support: Capability of the software to provide meaningful insights.

6.2 Best practices for evaluating digital twin performance

As noted by Peter Drucker, Mgt. consultant and author, “You cannot manage what you cannot measure.” This principle is equally applicable to digital twins. The confusion matrix employed in data science can measure digital twins' performance ARC Advisory Group ( 2024 ). Footnote 4 Assessing the performance of digital twins necessitates a thorough approach that considers multiple aspects, including hardware, data management middleware, and software components. Below are some essential practices for effectively assessing the performance of digital twins. The formulation of these best practices necessitated a thorough examination of the current literature on DT performance evaluation. Consultations with some stack holders were also conducted. By combining information from various sources, we hoped to convey the multidimensional nature of DT performance and provide meaningful advice to practitioners and researchers alike. Furthermore, the methods were iteratively refined to ensure their usefulness and applicability across various situations and industries.

Each practice recommended in this section is based on known management principles and its ability to address important difficulties in DT performance evaluation. For example, the emphasis on objective definition and particular Key Performance Indicators (KPIs) demonstrates the significance of goal alignment and measurement precision in achieving effective DT efforts. Similarly, data quality, security assessment, and scalability analysis methods emphasize these variables' importance in assuring the dependability and efficacy of distributed computing systems.

Objective definition: Clearly articulate the goals and objectives of the digital twin implementation to align performance indicators with broader business or operational objectives.

Establish specific KPIs: Identify and set specific Key Performance Indicators (KPIs) that align with the objectives, ensuring they are measurable, relevant, and linked to desired outcomes.

Multidimensional evaluation: Assess performance across multiple dimensions, including accuracy, responsiveness, scalability, security, and usability.

Regular review and update of metrics: Given the evolving nature of digital twin environments, performance metrics should be regularly reviewed and updated to maintain relevance and accuracy.

Focus on data quality and integrity: Emphasize metrics related to data accuracy, consistency, and integrity, as the quality of the digital twin largely depends on the reliability of its data.

Incorporate end-user experience metrics: Include metrics that gauge user satisfaction and adoption, such as visualization quality, interaction, and ease of use.

Measure latency and responsiveness: Evaluate latency in data collection, middleware processing, and software responsiveness to ensure real-time or near-real-time capabilities.

Security performance assessment: Implement metrics to evaluate the efficacy of security measures, including data encryption protocols.

Scalability analysis: Examine the digital twin's scalability, focusing on how well it accommodates increasing data volumes, user numbers, and processing requirements.

Simulation accuracy verification: Regularly validate the accuracy of simulations and virtual representations against actual world scenarios to ensure the digital twin's reliability.

Benchmarking: Compare performance against industry standards or best practices to understand how the digital twin stacks up against similar implementations.

Utilize monitoring technologies: Deploy monitoring technologies that offer real-time insights into the digital twin's operation, enabling proactive issue identification and resolution.

Develop a continuous improvement process: Establish a process for continuous improvement that integrates user feedback and ongoing evaluations, fostering a culture of perpetual enhancement.

By adhering to these practices, organizations can establish a robust framework for assessing and improving the performance of their digital twins, ensuring that these technologies deliver maximum value and effectively contribute to strategic objectives. To sum up, this section's performance indicators are the outcome of a systematic approach guided by academic research and industry observations. We hope to give readers a thorough grasp of how these measures support efficient DT evaluation procedures by outlining the reasoning behind their selection and their applicability to DT performance assessment.

7 Challenges associated with digital twins

Understanding the obstacles encountered while deploying digital twin technology is critical for its successful adoption and improvement. This section elucidates the difficulties various components of digital twin systems face, shedding light on their origins and implications. The challenges outlined are meticulously identified through an extensive review of literature and insights from field and industry experts, signifying their significance in the successful deployment and operation of digital twin systems. This analysis integrates multiple sources to pinpoint these hurdles as key challenges. The study organizes the identified challenges into three main aspects of digital twin technology: hardware, data management middleware, and software. This categorization facilitates a thorough understanding of the complex problems impacting different aspects of digital twin systems. A thorough examination of these challenges across the hardware, data management middleware, and software components aids in bridging the current research gap. Whereas prior studies often discussed these challenges in broad strokes, (Tuhaise et al. 2023 ) divided them into three categories: data transmission, interoperability, and data integration. This research details specific problems within each distinct component of the digital twin framework, thereby offering an in-depth analysis of the inherent obstacles in digital twins. It identifies hardware-related challenges, such as the complexity of sensor integration and issues with hardware reliability, suggesting solutions like adopting standardized sensor interfaces and employing predictive maintenance strategies. Furthermore, the study uncovers problems in data management middleware, including data integration bottlenecks and interoperability issues, recommending developing scalable middleware systems and adopting universal standards to enhance interoperability. The research outlines security vulnerabilities and algorithmic complexity regarding software components, proposing using advanced analytical tools and robust cybersecurity measures as solutions.

By delineating these issues across hardware, middleware, and software components, the study enhances the understanding of digital twin technology and offers actionable recommendations for enhancing the technology’s effectiveness and resilience. As digital twin technology continues to evolve, the findings underscore the necessity of concentrating on these components to surmount challenges and fully exploit the technology's potential across various applications and industries. The examination of digital twin elements and their associated challenges is visually summarized in Fig.  13 , which consists of three parts: (a) delineates the components of a digital twin, (b) identifies the challenges specific to each component, and (c) proposes solutions to these challenges.

Overview of digital twin components, associated challenges, and proposed solutions. Part ( a ) delineates the core components of a Digital Twin (DT). In part ( b ), a detailed breakdown highlights the challenges inherent in each component. Part ( c ) provides insightful solutions strategically proposed to address these challenges and enhance the effectiveness of Digital Twin implementation

7.1 Hardware components

Hardware components are the foundation of digital twin systems, comprising sensors, actuators, and other physical devices. Challenges within this component include:

Sensor integration complexity: Integrating diverse sensors for real-time data poses compatibility and synchronization issues.

Hardware reliability: Ensuring long-term sensor and actuator reliability is essential.

Proposed solutions involve adopting standardized sensor interfaces and implementing predictive maintenance strategies to mitigate these challenges.

7.2 Data management middleware

Middleware plays a crucial role in managing and processing the vast amount of data generated by digital twin systems. Challenges within this component include:

Data integration bottlenecks: Handling diverse data streams can lead to processing delays.

Interoperability issues: Different standards may hinder middleware system compatibility.

Proposed solutions include developing scalable middleware architectures and embracing industry-wide standards for improved interoperability.

7.3 Software components

Software components encompass the algorithms and analytical tools for real-time data analysis and decision-making. Challenges within this component include:

Algorithmic complexity: Complex algorithms for real-time analytics and decision-making need streamlining.

Security vulnerabilities: Software components are susceptible to cybersecurity threats.

Proposed solutions involve utilizing advanced analytical tools and robust cybersecurity protocols to address these challenges.

In conclusion, addressing the challenges linked with digital twins requires a deep understanding of their core components: hardware, data management middleware, and software. This analysis has unveiled various obstacles, from hardware constraints to data integration complexities and software interoperability challenges. A comprehensive perspective is provided by examining these issues across the distinct hardware, middleware, and software components. It is essential to identify and tackle the limitations associated with hardware, the challenges within middleware, and the issues related to software interoperability to enhance the efficiency and robustness of digital twin systems. As digital twin technology evolves, prioritizing these areas will be critical for navigating difficulties and leveraging the technology’s capacity in diverse applications and industries.

8 Case studies

Case studies in the realm of smart cities and digital twins serve as vital illustrations of these technologies in practical scenarios:

Dubai's "Happiness Agenda": A smart city initiative using big data to enhance urban living and measure "happiness" across various criteria. The objective was to involve every citizen in shaping future cities, particularly focusing on citizen engagement. Dubai’s "Happiness Agenda" implementation represents a notable example of a smart city involving its residents in urban development. Dubai has positioned itself as one of the "happiest" places to live by defining citizen "happiness" across multiple criteria. It uses big data analysis to allocate urban resources strategically, enhancing the city's overall "Happiness Index" (Zakzak 2019 ).

West Cambridge site and IFM building: These case studies explore adaptable digital twins at the building level, integrating various data sources and AI-driven decision-making. The West Cambridge site of the University of Cambridge in the UK was chosen as a case study due to its diverse facilities, which include university buildings, sports centers, residence areas, main roads, parking places, and restaurants. This variety allows for testing and evaluating the proposed dynamic digital twin system across different types of infrastructure. Additionally, the site's size and complexity offer an ideal environment to assess the effectiveness of the technology. Access to extensive data sources, collaboration opportunities with experts, and relevance to the academic community further contribute to its suitability as a testbed for the study (Qiuchen Lu et al. 2019 ).

Herrenberg, Germany: A case study demonstrating the use of digital twin technology in urban planning and city management. The case study of Herrenberg might illustrate the implementation and benefits of digital twins in improving urban planning, infrastructure management, and citizen engagement within the city. Herrenberg was selected as a case study for the digital twin due to its relevance to urban challenges, accessibility of diverse data sources, the potential for collaboration with local stakeholders, engagement of the community, and suitability in terms of size and complexity for testing the digital twin technology (Dembski et al. 2020 ).

Cambridge Sub-region: A digital twin pilot is developed, integrating diverse data streams for urban planning and decision-making. The authors stress the significance of including diverse data like IoT sensors, satellite images, social media, and government records to ensure an all-encompassing and precise city representation. The case study presented in the paper is about developing a digital twin pilot for the Cambridge Sub-region. It highlights how integrating various data streams and simulation models can assist urban planning, resource allocation, and decision-making processes. The case study provides insights into the potential benefits of using a city-level digital twin for improving efficiency, sustainability, and resilience in urban environments (Wan et al. 2019 ).

Málaga City: Implementing cognitive analytics in smart city management to enhance transportation, energy, and public services. The focus is on enhancing various aspects of urban life, such as transportation, energy management, waste management, and public services. The case study of Málaga City demonstrates the practical implementation of cognitive analytics to improve decision-making processes, optimize resource allocation, and ultimately enhance the quality of life for its residents (Pérez and Toledo 2017 ).

Ålesund, Norway: The study explores the role of a data-driven digital twin in enhancing urban systems and services within a smart city framework. It suggests using high-quality 3D graphical digital twins (GDTs) of cities to generate 4D visualizations of geolocalized time-series data to enhance citizen engagement. Through a case study conducted in Ålesund, Norway, the methodology utilizes readily available hardware and a game engine to develop immersive environments for presenting complex data sourced from GIS, BIM, demographics, and IoT. The approach emphasizes scalability, transferability, versatility in data integration, adherence to privacy regulations, and dependable data delivery. The paper introduces a pioneering smart city GDT framework, which capitalizes on interactive features and advancements in metrology (Major et al. 2021 ).

Case study in Greece: Details the development and application of digital twins tailored for smart cities, focusing on urban infrastructure improvements. The case study probably illustrates how digital twins optimize city systems, improve efficiency, and facilitate decision-making processes in Greek urban environments. This study might showcase practical examples of implementing digital twin technology to address challenges and enhance the overall functioning of a smart city in Greece (Evangelou et al. 2022 ).

Each case study offers unique insights into the deployment and impact of digital twin technology in various urban settings, highlighting its potential to improve city management and living standards. Table 2 offers an overview of each paper's focus areas, case studies, and key highlights, showcasing their distinct contributions and applications in the field of digital twins in smart cities.

9 Smart city governance in the era of digital twins: addressing challenges and leveraging opportunities

In the evolving discourse on smart cities and digital twin technologies, a critical examination of multi-level governance, organizational practices, and governance dimensions emerges as pivotal. The collective contributions from the referenced studies provide a comprehensive overview of the challenges and strategies in implementing smart city initiatives across different governance frameworks and geographical contexts.

As examined in one study, the integration of Chinese new authoritarian principles into smart government transitions highlights the inherent tensions between state-level directives and local-level implementation, underscoring the complexity of multi-level governance in authoritarian regimes (Zhang and Mora 2023 ). This perspective is enriched by a nuanced exploration of organizational practices within smart city development, revealing how bureaucratic, technocratic, and participatory logics intersect to shape decision-making and citizen engagement in smart city projects (Mora et al. 2023a ). Furthermore, the identification of three key governance dimensions—institutional context for urban innovation, urban innovation ecosystem, and urban digital innovation—provides a framework for understanding the governance mechanisms essential for fostering smart city transitions (Mora et al. 2023b ).

Critical analysis across the studies reveals common challenges in smart city governance, such as interoperability and compatibility issues within the digital ecosystem and integrating a technological dimension in urban development. These challenges underscore the importance of addressing interoperability and compatibility to enhance city planning and management effectively (Quek et al. 2023 ). The discourse extends to the critical analysis of smart urbanism in non-Western contexts, notably in India and Africa, where issues of urban informality, equity, and the inclusivity of smart city initiatives are brought to the forefront (Prasad et al. 2023 ; Tonnarelli and Mora 2023 ). These analyses highlight the necessity of adopting equitable and inclusive smart city development approaches that consider the needs and priorities of all urban dwellers, particularly marginalized communities.

Moreover, the call for empirical studies and the integration of innovation management theory into smart city governance research emphasizes the need for practical guidance and theoretical advancements in managing urban digital innovation (Mora et al. 2023b ). The exploration of human-cyber-physical interactions further illuminates the evolving relationship between technology, governance, and societal dynamics, advocating for a holistic approach that balances technological advancements with ethical and sociocultural considerations (Quek et al. 2023 ).

In conclusion, the amalgamated insights from these studies advocate for a pragmatic, contextually informed, and inclusive approach to smart city governance. By addressing the multifaceted challenges of interoperability, governance, and citizen engagement, and by incorporating a critical perspective on urban informality and inclusivity, this body of work contributes significantly to the scholarly discourse on smart cities and digital twins. The emphasis on empirical research, innovation management, and the integration of technology in urban development underscores the dynamic interplay between technology, governance, and urban development strategies in the quest for sustainable and equitable urban futures.

9.1 Role of DT in smart city governance

Smart city governance constitutes a complex framework fundamental to the effective realization and long-term viability of smart city endeavors. It encompasses the strategic alignment of policies, technological systems, and multifaceted collaborations amongst stakeholders by overarching urban development goals. Digital twin technology plays a pivotal role in enhancing smart city governance by offering innovative solutions across various components:

Policy and strategy formulation : Crafting policies and strategies that guide smart city initiatives in service of the city's broader objectives (Beckers 2022). Digital twins assist in crafting policies and strategies by providing valuable insights derived from real-time data and simulations. City authorities can utilize digital twins to assess the impact of different policies and strategies on urban systems, enabling informed decision-making aligned with broader city objectives.

Collaborative ecosystem : Fostering partnerships spanning government entities, the private sector, academic institutions, and the citizenry, thus leveraging collective knowledge and resources (Beckers 2022). Digital twins foster collaboration among government agencies, private sector entities, academic institutions, and citizens by providing a platform for data sharing and analysis. This collaborative ecosystem enhances collective knowledge and resource utilization, facilitating more effective governance practices and co-creating solutions to urban challenges.

Technological infrastructure : Establishing and administering the technological basis, encompassing data management and digital platforms, that underpins smart city operations (Zhang and Mora 2023 ). As a foundational element of smart city operations, digital twins contribute to establishing and managing the technological infrastructure required for governance. They enable comprehensive data management and visualization, empowering city administrators to monitor urban systems, identify emerging trends, and respond proactively to issues in real-time.

Ethical considerations : Prioritizing ethical concerns by safeguarding data privacy and security and ensuring the equitable deployment of technology (Mora et al. 2023a ). Digital twins support ethical governance by prioritizing data privacy, security, and equitable technology deployment. Through robust data encryption protocols and access controls, digital twins safeguard sensitive information, ensuring that governance processes remain transparent, accountable, and inclusive for all stakeholders.

Public participation : Stimulating citizen involvement in the governance process promotes transparency and inclusiveness (Mora et al. 2023b ). Digital twins facilitate public participation by providing accessible platforms for citizen feedback, collaboration, and co-design of urban solutions. By incorporating citizen inputs into decision-making processes, digital twins help ensure that governance strategies align with community needs and preferences.

Sustainability : Championing sustainable development practices integrated within smart city projects to prioritize environmental stewardship and long-term resilience (Quek et al. 2023 ). By simulating various scenarios and assessing the environmental impact of proposed policies and projects, digital twins enable city authorities to prioritize sustainability and resilience in urban planning and decision-making processes.

9.2 Smart city governance challenges

The pursuit of smart city objectives is frequently hindered by governance crises, underscoring the complexities of managing urban digital transformations. Digital twins offer innovative solutions to navigate the complexities of urban governance and enhance decision-making processes. Here, we explore how digital twins can be utilized to tackle key challenges in smart city governance:

Data privacy and security concerns: Contending with data privacy and security risks associated with the vast collection and storage of urban data (Mora et al. 2023a ). Digital twins incorporate robust data encryption protocols and access controls, ensuring the protection of sensitive information within smart city systems. Digital twins help mitigate privacy and security risks associated with urban data collection and storage by enabling secure data management and transmission.

Digital divide and inequity: Mitigating the digital divide can potentially intensify social disparities within urban communities (Prasad et al. 2023 ). Digital twins promote inclusivity and bridge the digital divide by providing accessible platforms for citizen engagement and participation in governance processes. Through user-friendly interfaces and interactive visualization tools, digital twins empower all citizens to contribute to decision-making, regardless of their technological literacy or socioeconomic status.

Regulatory and legal challenges: Navigating the disparity between the swift pace of technological progress and prevailing regulatory frameworks (Tonnarelli and Mora 2023 ). Digital twins assist city authorities in navigating regulatory and legal frameworks by providing comprehensive data analytics and scenario modeling capabilities. Digital twins facilitate informed policy-making and ensure alignment with legal standards and industry regulations by simulating the impact of proposed regulations and assessing compliance requirements.

Fragmented governance structures: Surmounting the intricacies of multi-stakeholder governance structures, which can obstruct coordinated action (Zhang and Mora 2023 ). Digital twins serve as centralized platforms for data integration and collaboration, overcoming the challenges of fragmented governance structures. By consolidating diverse datasets from multiple stakeholders and domains, digital twins enable seamless information sharing and coordination, fostering synergy among various governmental entities and stakeholders.

Resource constraints: Confronting limitations in financial, technical, and operational capacities is vital to the success of smart city ventures (Quek et al. 2023 ). Digital twins optimize resource utilization and operational efficiencies within smart city governance through predictive analytics and optimization algorithms. By identifying inefficiencies and optimizing resource allocation, digital twins help cities overcome resource constraints and maximize the impact of limited financial, technical, and operational resources.

By integrating digital twin technologies, smart city administration can address these challenges, paving the way for innovative solutions and sustainable urban development. Digital twins offer a comprehensive and data-driven approach to governance, enabling cities to enhance decision-making processes, accountability, and transparency, ultimately enhancing the quality of life for urban residents. While digital twin technologies have the potential to significantly improve urban management through advanced data analytics, simulation, and optimization, their seamless integration into smart city governance requires careful consideration of governance issues. This includes addressing concerns related to data privacy, fostering collaboration among stakeholders, and upholding ethical principles.

10 Conclusions and future research directions

This survey paper employs a meticulous bibliometric methodology, selecting the Web of Science database for its comprehensive coverage and developing precise search criteria to gather over 4,220 relevant articles. The analysis uses advanced tools like VOSviewer for network analyses and visualizations, including co-authorship and keyword co-occurrence maps, enabling a detailed examination of trends and relationships in Digital Twin technology and Smart Cities research. This methodological rigor ensures the study's reliability and contributes to its uniqueness in the field. This survey comprehensively reviews over 4200 publications in the domain of Digital Twins and Smart Cities. It outlines the evolution, applications, and integration of Digital Twins with IoT and AI in urban development. The survey distinguishes itself through extensive bibliometric analysis, focusing on datasets, platforms, software, and performance metrics, and it offers unique insights into the challenges and opportunities within the field. The findings include emerging trends, key thematic areas, and a detailed exploration of various Smart City applications. The paper concludes with implications for urban developers, policymakers, and researchers and recommendations for future research directions. The field of Digital Twin (DT) and Smart Cities is ripe for future research, aiming to overcome current challenges and explore new frontiers. Detailed investigation and development in this area are essential for realizing the full potential of DT technologies in urban environments. The discussions pave the way for sustainable and equitable urban futures, recognizing the dynamic interplay between technology, governance, and urban development strategies.

Future research should focus on:

Enhanced data integration : Developing more efficient methods for integrating diverse data sources within DT systems.

Scalability solutions : Creating scalable DT models suitable for larger and more complex urban environments.

Advanced security protocols : Strengthening cybersecurity measures for DT systems to ensure data privacy and security.

Sophisticated analytical tools : Incorporating cutting-edge AI and machine learning techniques for predictive analytics and decision-making.

Expanding IoT capabilities : Extending the use of IoT in DTs for comprehensive real-time data collection and monitoring.

Sustainable urban development : Leveraging DTs for resource management, focusing on sustainability and environmental conservation.

Citizen engagement models : Developing DTs prioritizing citizen involvement in urban planning and management.

Policy and governance studies : Examining the influence of policy in guiding DT implementation and addressing ethical concerns.

Economic impact assessment : Evaluating the economic implications of DTs, including cost analysis and return on investment.

Real-world case studies : Documenting extensive case studies to assess DTs' practical impact and challenges in urban settings.

Investigating future technological advancements: new applications, and the role of policy and governance in Digital Twins development.

The findings of this paper are poised to influence future research, policy-making, and practical applications in Smart Cities and Digital Twins in significant ways:

Informing future research directions: The comprehensive review of over 4,200 publications provides valuable insights into the current state of Digital Twins and Smart Cities research. Researchers can utilize this information to identify gaps in existing literature and prioritize areas for further investigation. For example, identifying challenges such as data integration bottlenecks and security vulnerabilities can guide future research efforts toward developing solutions to these pressing issues.

Guiding policy development: Policymakers can leverage the findings of this paper to inform the development of policies and regulations related to Digital Twin technology and its application in Smart Cities. By understanding the challenges and opportunities associated with Digital Twins, policymakers can create frameworks that promote innovation while addressing data privacy, cybersecurity, and ethical considerations.

Improving urban planning and management: The insights provided by this paper can assist urban planners and city managers in making informed decisions about adopting and implementing Digital Twins in Smart Cities. By understanding Digital Twin technology's potential benefits and challenges, city officials can develop strategies to optimize urban infrastructure, improve resource management, and enhance citizen services.

Driving technological innovation: The paper identifies emerging trends and technological advancements in Digital Twin technology, such as the integration of AI and IoT, as well as the development of scalable models and advanced security protocols. These insights can inspire innovation in academia and industry, leading to the development of new tools, platforms, and solutions that push the boundaries of Digital Twin technology and its applications in Smart Cities.

Finally, the findings of this article have the potential to spark advances in research, policymaking, and practical applications connected to Digital Twins and Smart Cities, resulting in more efficient, sustainable, and resilient urban development.

Data availability

Data is provided within the manuscript.

http://www.odaa.dk

https://data.sfgov.org

https://data.cityofchicago.org

https://www.arcweb.com/industry-best-practices/measuring-digital-twin-performance-maturity-confusion-matrix

Abdeen FN, Shirowzhan S, Sepasgozar SME (2023) Citizen-centric digital twin development with machine learning and interfaces for maintaining urban infrastructure. Telematics Inform 84:102032. https://doi.org/10.1016/j.tele.2023.102032

Article   Google Scholar  

Abideen AZ, Sundram VPK, Pyeman J, Othman AK, Sorooshian S (2021) Digital twin integrated reinforced learning in supply chain and logistics. Logistics 5(4):84. https://doi.org/10.3390/logistics5040084

Abouzid I, Saidi R (2023) Digital twin implementation approach in supply chain processes. Sci Afr 21:e01821. https://doi.org/10.1016/j.sciaf.2023.e01821

Aghdam ZN, Rahmani AM, Hosseinzadeh M (2021) The role of the internet of things in healthcare: future trends and challenges. Comput Methods Programs Biomed 199:105903. https://doi.org/10.1016/j.cmpb.2020.105903

Aheleroff S, Zhong RY, Xu X, Feng Z, Goyal P (2020) Digital twin enabled mass personalization: a case study of a smart wetland maintenance system. Volume 2: manufacturing processes; manufacturing systems; nano/micro/meso manufacturing; quality and reliability. https://doi.org/10.1115/msec2020-8363

Akroyd J, Harper Z, Soutar D, Farazi F, Bhave A, Mosbach S, Kraft M (2022) Universal digital twin: land use. Data-Centric Eng 3. https://doi.org/10.1017/dce.2021.21

Alam KM, El Saddik A (2017) C2PS: a digital twin architecture reference model for the cloud-based cyber-physical systems. IEEE Access 5:2050–2062. https://doi.org/10.1109/access.2017.2657006

Allam Z, Jones DS (2021) Future (post-COVID) digital, smart and sustainable cities in the wake of 6G: digital twins, immersive realities and new urban economies. Land Use Policy 101:105201. https://doi.org/10.1016/j.landusepol.2020.105201

Al-Obaidy AHMJ, Khalaf SM, Hassan FM (2022) Application of CCME index to assess the water quality of tigris river within Baghdad City, Iraq. IOP Conf Ser: Earth Environ Sci 1088(1):012004. https://doi.org/10.1088/1755-1315/1088/1/012004

Amtsberg F, Yang X, Skoury L, Wagner H-J, Menges A (2021) iHRC: an AR-based interface for intuitive, interactive and coordinated task sharing between humans and robots in building construction. Proceedings of the International Symposium on Automation and Robotics in Construction (IAARC). https://doi.org/10.22260/isarc2021/0006

ARC Advisory Group (2024) Measuring digital twin performance maturity with the confusion matrix. Retrieved from https://www.arcweb.com/industry-best-practices/measuring-digital-twin-performance-maturity-confusion-matrix . Accessed 10 May 2024

Ariyachandra MRMF, Wedawatta G (2023) Digital twin smart cities for disaster risk management: a review of evolving concepts. Sustainability 15(15):11910. https://doi.org/10.3390/su151511910

ASME (2018) Retracted: “Kinematic analysis of the motion of a six degrees of freedom wave energy converter based on the concept of the stewart-gough platform” [ASME 2018 37th international conference on ocean, offshore and arctic engineering, Volume 10: ocean renewable energy, Madrid, Spain, June 17–22, 2018, Conference sponsors: ocean, offshore and arctic engineering division, ISBN: 978–0–7918–5131–9. Paper No. OMAE2018–78601, pp V010T09A047; 10 pages]. Volume 10: ocean renewable energy. https://doi.org/10.1115/OMAE2018-78601

Autodesk Revit Architecture Certification (2019) Mastering Autodesk® Revit® 2020, 1033–1035. Portico. https://doi.org/10.1002/9781119570189.app3

Badwi IM, Ellaithy HM, Youssef HE (2022) 3D-GIS parametric modelling for virtual urban simulation using CityEngine. Ann GIS 28(3):325–341. https://doi.org/10.1080/19475683.2022.2037019

Bagaria N, Laamarti F, Badawi HF, Albraikan A, Martinez Velazquez RA, El Saddik A (2019) Health 4.0: digital twins for health and well-being. Connected health in smart cities. pp 143–152. https://doi.org/10.1007/978-3-030-27844-1_7

Bellavista P, Giannelli C, Mamei M, Mendula M, Picone M (2021) Application-driven network-aware digital twin management in industrial edge environments. IEEE Trans Industr Inf 17(11):7791–7801. https://doi.org/10.1109/tii.2021.3067447

Biagini V, Subasic M, Oudalov A, Kreusel J (2020) The autonomous grid: automation, intelligence and the future of power systems. Energy Res Soc Sci 65:101460. https://doi.org/10.1016/j.erss.2020.101460

Billinghurst M, Clark A, Lee G (2015) A survey of augmented reality. Foundations and Trends® in Human–Computer Interaction, 8(2–3):73–272. https://doi.org/10.1561/1100000049

Blomkvist J, Clatworthy S, Holmlid S (2023) Interlude 1: materiality in design from a practitioner perspective: interview with Markus Edgar Hormeß, Adam Lawrence and Marc Stickdorn (26 April 2022). The materials of service design. pp 83–88. https://doi.org/10.4337/9781802203301.00019

Borrmann A, König M, Koch C, Beetz J (2018). Building information modeling: why? What? How? Building information modeling. pp 1–24. https://doi.org/10.1007/978-3-319-92862-3_1

Bouzguenda I, Alalouch C, Fava N (2019) Towards smart sustainable cities: a review of the role digital citizen participation could play in advancing social sustainability. Sustain Cities Soc 50:101627. https://doi.org/10.1016/j.scs.2019.101627

Catalano M, Chiurco A, Fusto C, Gazzaneo L, Longo F, Mirabelli G, Nicoletti L, Solina V, Talarico S (2022) A digital twin-driven and conceptual framework for enabling extended reality applications: a case study of a brake discs manufacturer. Procedia Comput Sci 200:1885–1893. https://doi.org/10.1016/j.procs.2022.01.389

Cathey G, Benson J, Gupta M, Sandhu R (2021) Edge centric secure data sharing with digital twins in smart ecosystems. 2021 Third IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA). https://doi.org/10.1109/tpsisa52974.2021.00008

Cerovsek T (2011) A review and outlook for a ‘Building Information Model’ (BIM): a multi-standpoint framework for technological development. Adv Eng Inform 25(2):224–244. https://doi.org/10.1016/j.aei.2010.06.003

Chakraborty S, Adhikari S (2021) Machine learning based digital twin for dynamical systems with multiple time-scales. Comput Struct 243:106410. https://doi.org/10.1016/j.compstruc.2020.106410

Chang T-W, Hsiao C-F, Chen C-Y, Huang W-X, Datta S, Mao W-L (2020) Fabricating behavior sensor computing approach for coexisting design environment. Sensors Mater 32(7):2409. https://doi.org/10.18494/sam.2020.2809

Charitonidou M (2022) Urban scale digital twins in data-driven society: challenging digital universalism in urban planning decision-making. Int J Archit Comput 20(2):238–253. https://doi.org/10.1177/14780771211070005

Chen L, Xie X, Lu Q, Parlikad AK, Pitt M, Yang J (2021) Gemini principles-based digital twin maturity model for asset management. Sustainability 13(15):8224. https://doi.org/10.3390/su13158224

Cinar ZM, Nuhu AA, Zeeshan Q, Korhan O (2020) Digital twins for industry 4.0: a review. Industrial engineering in the digital disruption era, 193–203. https://doi.org/10.1007/978-3-030-42416-9_18

City of Chicago Data Portal (2024) Chicago Data Portal. https://data.cityofchicago.org . Accessed 10 May 2024

Crespi N, Drobot AT, Minerva R (2023) The digital twin: what and why? The digital twin.pp 3–20. https://doi.org/10.1007/978-3-031-21343-4_1

Damjanovic-Behrendt V, Behrendt W (2019) An open source approach to the design and implementation of Digital Twins for Smart Manufacturing. Int J Comput Integr Manuf 32(4–5):366–384. https://doi.org/10.1080/0951192x.2019.1599436

Damjanovic-Behrendt V (2018) A digital twin-based privacy enhancement mechanism for the automotive industry. 2018 International Conference on Intelligent Systems (IS). https://doi.org/10.1109/is.2018.8710526

Dani AAH, Supangkat SH, Lubis FF, Nugraha IGBB, Kinanda R, Rizkia I (2023) Development of a smart city platform based on digital twin technology for monitoring and supporting decision-making. Sustainability 15(18):14002. https://doi.org/10.3390/su151814002

Defraeye T, Tagliavini G, Wu W, Prawiranto K, Schudel S, Assefa Kerisima M, Verboven P, Bühlmann A (2019) Digital twins probe into food cooling and biochemical quality changes for reducing losses in refrigerated supply chains. Resour Conserv Recycl 149:778–794. https://doi.org/10.1016/j.resconrec.2019.06.002

Dembski F, Wössner U, Letzgus M, Ruddat M, Yamu C (2020) Urban digital twins for smart cities and citizens: the case study of Herrenberg, Germany. Sustainability 12(6):2307. https://doi.org/10.3390/su12062307

Deren L, Wenbo Y, Zhenfeng S (2021) Smart city based on digital twins. Comput Urban Sci 1(1). https://doi.org/10.1007/s43762-021-00005-y

Diakite AA, Ng L, Barton J, Rigby M, Williams K, Barr S, Zlatanova S (2022) Liveable city digital twin: a pilot project for the city of Liverpool (NSW, Australia). ISPRS Ann Photogramm Remote Sens Spatial Inf Sci X-4/W2-2022:45–52. https://doi.org/10.5194/isprs-annals-x-4-w2-2022-45-2022

Dimitropoulos N, Togias T, Zacharaki N, Michalos G, Makris S (2021) Seamless human-robot collaborative assembly using artificial intelligence and wearable devices. Appl Sci 11(12):5699. https://doi.org/10.3390/app11125699

Din GMU, Marnerides AK (2017) Short term power load forecasting using Deep Neural Networks. 2017 International Conference on Computing, Networking and Communications (ICNC). https://doi.org/10.1109/iccnc.2017.7876196

Dong R, She C, Hardjawana W, Li Y, Vucetic B (2019) Deep learning for hybrid 5G services in mobile edge computing systems: learn from a digital twin. IEEE Trans Wireless Commun 18(10):4692–4707. https://doi.org/10.1109/twc.2019.2927312

Endsley MR (2016) Designing for situation awareness. CRC Press. https://doi.org/10.1201/b11371

Erol T, Mendi AF, Dogan D (2020) The digital twin revolution in healthcare. 2020 4th International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT). https://doi.org/10.1109/ismsit50672.2020.9255249

Evangelou T, Gkeli M, Potsiou C (2022) Building digital twins for smart cities: a case study in Greece. ISPRS Ann Photogramm Remote Sens Spatial Inf Sci X-4/W2-2022:61–68. https://doi.org/10.5194/isprs-annals-x-4-w2-2022-61-2022

Fan C, Mostafavi A (2019) A graph-based method for social sensing of infrastructure disruptions in disasters. Comput-Aided Civ Infrastruct Eng 34(12):1055–1070. https://doi.org/10.1111/mice.12457 . (Portico)

Fan C, Zhang C, Yahja A, Mostafavi A (2021) Disaster City Digital Twin: a vision for integrating artificial and human intelligence for disaster management. Int J Inf Manage 56:102049. https://doi.org/10.1016/j.ijinfomgt.2019.102049

Fan C, Jiang Y, Mostafavi A (2020) Social sensing in disaster city digital twin: integrated textual–visual–geo framework for situational awareness during built environment disruptions. J Manag Eng 36(3). https://doi.org/10.1061/(asce)me.1943-5479.0000745

Fang X, Wang H, Liu G, Tian X, Ding G, Zhang H (2022) Industry application of digital twin: from concept to implementation. Int J Adv Manuf Technol 121(7–8):4289–4312. https://doi.org/10.1007/s00170-022-09632-z

Ford DN, Wolf CM (2020) Smart cities with digital twin systems for disaster management. J Manag Eng 36(4). https://doi.org/10.1061/(asce)me.1943-5479.0000779

Gerlach B, Zarnitz S, Nitsche B, Straube F (2021) Digital supply chain twins—conceptual clarification, use cases and benefits. Logistics 5(4):86. https://doi.org/10.3390/logistics5040086

Ghaith M, Yosri A, El-Dakhakhni W (2022) Digital twin: a city-scale flood imitation framework. Proceedings of the Canadian society of civil engineering annual conference 2021. pp 577–588. https://doi.org/10.1007/978-981-19-1065-4_48

Ghosh D, Chun SA, Shafiq B, Adam NR (2016) Big data-based smart city platform. Proceedings of the 17th international digital government research conference on digital government research. https://doi.org/10.1145/2912160.2912205

Goia B, Cioara T, Anghel I (2022) Virtual power plant optimization in smart grids: a narrative review. Future Internet 14(5):128. https://doi.org/10.3390/fi14050128

Grieves MW (2005) Product lifecycle management: the new paradigm for enterprises. Int J Prod Dev 2(1/2):71. https://doi.org/10.1504/ijpd.2005.006669

Guarino N, Oberle D, Staab S (2009) What is an ontology? Handbook on ontologies. pp 1–17. https://doi.org/10.1007/978-3-540-92673-3_0

Guo D, Zhong RY, Lin P, Lyu Z, Rong Y, Huang GQ (2020) Digital twin-enabled Graduation Intelligent Manufacturing System for fixed-position assembly islands. Robot Comput-Integr Manuf 63:101917. https://doi.org/10.1016/j.rcim.2019.101917

He X, Ai Q, Wang J, Tao F, Pan B, Qiu R, Yang B (2023) Situation awareness of energy internet of things in smart city based on digital twin: from digitization to informatization. IEEE Internet Things J 10(9):7439–7458. https://doi.org/10.1109/jiot.2022.3203823

He Y, Deng J, Li H (2017) Short-term power load forecasting with deep belief network and copula models. 2017 9th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC). https://doi.org/10.1109/ihmsc.2017.50

He X, Ai Q, Qiu RC, Zhang D (2019) Preliminary exploration on digital twin for power systems: challenges, framework, and applications. arXiv preprint arXiv:1909.06977. https://www.semanticscholar.org/paper/Preliminary-Exploration-on-Digital-Twin-for-Power-He-Ai/360cd3a26ad4ab5f3685d676d36001348dd6e1d6 . Accessed 10 May 2024

Hinchy EP, O’Dowd NP, McCarthy CT (2019) Using open-source microcontrollers to enable digital twin communication for smart manufacturing. Procedia Manuf 38:1213–1219. https://doi.org/10.1016/j.promfg.2020.01.212

Hu L, Nguyen N-T, Tao W, Leu MC, Liu XF, Shahriar MR, Al Sunny SMN (2018) Modeling of cloud-based digital twins for smart manufacturing with MT connect. Procedia Manuf 26:1193–1203. https://doi.org/10.1016/j.promfg.2018.07.155

Huang J, Zhao L, Wei F, Cao B (2021) The application of digital twin on power industry. IOP Conf Ser: Earth Environ Sci 647(1):012015. https://doi.org/10.1088/1755-1315/647/1/012015

Ivanov S, Nikolskaya K, Radchenko G, Sokolinsky L, Zymbler M (2020) Digital twin of city: concept overview. 2020 Global Smart Industry Conference (GloSIC). https://doi.org/10.1109/glosic50886.2020.9267879

Jaensch F, Csiszar A, Scheifele C, Verl A (2018) Digital twins of manufacturing systems as a base for machine learning. 2018 25th International Conference on Mechatronics and Machine Vision in Practice (M2VIP). https://doi.org/10.1109/m2vip.2018.8600844

Jafari M, Kavousi-Fard A, Chen T, Karimi M (2023) A review on digital twin technology in smart grid, transportation system and smart city: challenges and future. IEEE Access 11:17471–17484. https://doi.org/10.1109/access.2023.3241588

Johansen ST, Unal P, Albayrak Ö, Ikonen E, Linnestad KJ, Jawahery S, Srivastava AK, Løvfall BT (2023) Hybrid and cognitive digital twins for the process industry. Open Eng 13(1). https://doi.org/10.1515/eng-2022-0418

John Samuel I, Salem O, He S (2022) Defect-oriented supportive bridge inspection system featuring building information modeling and augmented reality. Innov Infrastruct Solut 7(4). https://doi.org/10.1007/s41062-022-00847-3

Jones D, Snider C, Nassehi A, Yon J, Hicks B (2020) Characterising the Digital Twin: a systematic literature review. CIRP J Manuf Sci Technol 29:36–52. https://doi.org/10.1016/j.cirpj.2020.02.002

Kapteyn MG, Knezevic DJ, Huynh DBP, Tran M, Willcox KE (2020) Data-driven physics-based digital twins via a library of component-based reduced-order models. Int J Numer Methods Eng 123(13):2986–3003. https://doi.org/10.1002/nme.6423 . (Portico)

Ketzler B, Naserentin V, Latino F, Zangelidis C, Thuvander L, Logg A (2020) Digital twins for cities: a state of the art review. Built Environ 46(4):547–573. https://doi.org/10.2148/benv.46.4.547

Khudhair A, Li H, Ren G, Liu S (2021) Towards future BIM technology innovations: a bibliometric analysis of the literature. Appl Sci 11(3):1232. https://doi.org/10.3390/app11031232

Kim J, Olsen D (2021) From BIM to inspection: a comparative analysis of assistive embedment inspection. Proceedings of the International Symposium on Automation and Robotics in Construction (IAARC). https://doi.org/10.22260/isarc2021/0123

Kitchenham B, Pearl Brereton O, Budgen D, Turner M, Bailey J, Linkman S (2009) Systematic literature reviews in software engineering – a systematic literature review. Inf Softw Technol 51(1):7–15. https://doi.org/10.1016/j.infsof.2008.09.009

Koteleva N, Valnev V, Frenkel I (2021) Investigation of the effectiveness of an augmented reality and a dynamic simulation system collaboration in oil pump maintenance. Appl Sci 12(1):350. https://doi.org/10.3390/app12010350

Kritzinger W, Karner M, Traar G, Henjes J, Sihn W (2018) Digital Twin in manufacturing: a categorical literature review and classification. IFAC-PapersOnLine 51(11):1016–1022. https://doi.org/10.1016/j.ifacol.2018.08.474

Kritzler M, Funk M, Michahelles F, Rohde W (2017) The virtual twin. Proceedings of the seventh international conference on the internet of things. https://doi.org/10.1145/3131542.3140274

Krzyczkowski D (2019) Introducing azure digital twins. Apress. https://doi.org/10.1007/978-1-4842-5375-5

Kuber S, Sharma M, Bonetti A, Harispuru C, Soroush A (2022) Virtual testing of protection systems using digital twin technology. 2022 75th Annual Conference for Protective Relay Engineers (CPRE). https://doi.org/10.1109/cpre55809.2022.9776572

Kyriazopoulou C (2015) Smart city technologies and architectures - a literature review. Proceedings of the 4th International Conference on Smart Cities and Green ICT Systems. https://doi.org/10.5220/0005407000050016

Lacoche J, Villain É (2022) Prototyping context-aware augmented reality applications for smart environments inside virtual reality. Proceedings of the 17th international joint conference on computer vision, imaging and computer graphics theory and applications. https://doi.org/10.5220/0010768800003124

Lam P-D, Han J, Kwon K-R, Ok S-Y, Lee S-H (2023) Semantic 3D city model data visualization for smar t city digital twin. J Korea Multimed Soc 26(2):116–130. https://doi.org/10.9717/kmms.2023.26.2.116

Langenheim N, Sabri S, Chen Y, Kesmanis A, Felson A, Mueller A, Rajabifard A, Zhang Y (2022) Adapting a digital twin to enable real-time water sensitive urban design decision-making. Int Arch Photogramm Remote Sens Spat Inf Sci XLVIII-4/W4-2022:95–100. https://doi.org/10.5194/isprs-archives-xlviii-4-w4-2022-95-2022

Ledoux H, Arroyo Ohori K, Kumar K, Dukai B, Labetski A, Vitalis S (2019) CityJSON: a compact and easy-to-use encoding of the CityGML data model. Open Geospatial Data Softw Standards 4(1). https://doi.org/10.1186/s40965-019-0064-0

Lee A, Lee K-W, Kim K-H, Shin S-W (2022) A geospatial platform to manage large-scale individual mobility for an urban digital twin platform. Remote Sens 14(3):723. https://doi.org/10.3390/rs14030723

Lei B, Stouffs R, Biljecki F (2022) Assessing and benchmarking 3D city models. Int J Geogr Inf Sci 37(4):788–809. https://doi.org/10.1080/13658816.2022.2140808

Lemos MR, Cardoso VF, Otani M, Da Costa Nunes R, Da Silva VJ, De Lucena Junior VF (2022) Navigation robot training with Deep Q-Learning monitored by Digital Twin. 2022 IEEE International Conference on Consumer Electronics (ICCE). https://doi.org/10.1109/icce53296.2022.9730282

Li W, Rentemeister M, Badeda J, Jöst D, Schulte D, Sauer DU (2020) Digital twin for battery systems: cloud battery management system with online state-of-charge and state-of-health estimation. J Energy Storage 30:101557. https://doi.org/10.1016/j.est.2020.101557

Li C, Zheng P, Li S, Pang Y, Lee CKM (2022) AR-assisted digital twin-enabled robot collaborative manufacturing system with human-in-the-loop. Robot Comput-Integr Manuf 76:102321. https://doi.org/10.1016/j.rcim.2022.102321

Li Y, Zhang Z, Li X, Guan S (2021) Research on equipment maintenance guidance technology based on MR and digital twin. Proceedings of the 2021 5th international conference on electronic information technology and computer engineering. https://doi.org/10.1145/3501409.3501454

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, Clarke M, Devereaux PJ, Kleijnen J, Moher D (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol 62(10):e1–e34. https://doi.org/10.1016/j.jclinepi.2009.06.006

Liu Y, Zhang L, Yang Y, Zhou L, Ren L, Wang F, Liu R, Pang Z, Deen MJ (2019) A novel cloud-based framework for the elderly healthcare services using digital twin. IEEE Access 7:49088–49101. https://doi.org/10.1109/access.2019.2909828

Liu S, Wang XV, Wang L (2022) Digital twin-enabled advance execution for human-robot collaborative assembly. CIRP Ann 71(1):25–28. https://doi.org/10.1016/j.cirp.2022.03.024

Liu Y, Pan S, Folz P, Ramparany F, Bolle S, Ballot E, Coupaye T (2023) Cognitive digital twins for freight parking management in last mile delivery under smart cities paradigm. Comput Ind 153:104022. https://doi.org/10.1016/j.compind.2023.104022

Liu Y, Wang Z, Han K, Shou Z, Tiwari P, L. Hansen JH (2020) Sensor fusion of camera and cloud digital twin information for intelligent vehicles. 2020 IEEE intelligent vehicles symposium (IV). https://doi.org/10.1109/iv47402.2020.9304643

Lohtander M, Ahonen N, Lanz M, Ratava J, Kaakkunen J (2018) Micro manufacturing unit and the corresponding 3D-model for the digital twin. Procedia Manuf 25:55–61. https://doi.org/10.1016/j.promfg.2018.06.057

Lu Y, Huang X, Zhang K, Maharjan S, Zhang Y (2021a) Low-latency federated learning and blockchain for edge association in digital twin empowered 6G networks. IEEE Trans Industr Inf 17(7):5098–5107. https://doi.org/10.1109/tii.2020.3017668

Lu Y, Maharjan S, Zhang Y (2021b) Adaptive edge association for wireless digital twin networks in 6G. IEEE Internet Things J 8(22):16219–16230. https://doi.org/10.1109/jiot.2021.3098508

Lu Q, Parlikad AK, Woodall P, Don Ranasinghe G, Xie X, Liang Z, Konstantinou E, Heaton J, Schooling J (2020) Developing a digital twin at building and city levels: case study of West Cambridge Campus. J Manag Eng 36(3). https://doi.org/10.1061/(asce)me.1943-5479.0000763

Lugaresi G, Jemai Z, Sahin E (2023) Digital twins for supply chains: current outlook and future challenges. ECMS 2023 Proceedings Edited by Enrico Vicario, Romeo Bandinelli, Virginia Fani, Michele Mastroianni. https://doi.org/10.7148/2023-0451

Mainisa M, Eka Priana S, Zulhedi Z (2023) Implementasi bim dalam permodelan 3d pembangunan gedung kantor cabang bri batusangkar menggunakan software openbuildings designer. Ensiklopedia Res Community Serv Rev 2(3):147–155. https://doi.org/10.33559/err.v2i3.1771

Major P, Li G, Hildre HP, Zhang H (2021) The use of a data-driven digital twin of a smart city: a case study of Ålesund, Norway. IEEE Instrum Meas Mag 24(7):39–49. https://doi.org/10.1109/mim.2021.9549127

Mandl B, Stehling M, Schmiedinger T, Adam M (2017) Enhancing workplace learning by augmented reality. Proceedings of the seventh international conference on the internet of things. https://doi.org/10.1145/3131542.3140265

Martinez-Velazquez R, Gamez R, El Saddik A (2019) Cardio twin: a digital twin of the human heart running on the edge. 2019 IEEE International Symposium on Medical Measurements and Applications (MeMeA). https://doi.org/10.1109/memea.2019.8802162

Meier N, Müller-Polyzou R, Brach L, Georgiadis A (2021) Digital twin support for laser-based assembly assistance. Procedia CIRP 99:460–465. https://doi.org/10.1016/j.procir.2021.03.066

Mihai S, Yaqoob M, Hung DV, Davis W, Towakel P, Raza M, Karamanoglu M, Barn B, Shetve D, Prasad RV, Venkataraman H, Trestian R, Nguyen HX (2022) Digital twins: a survey on enabling technologies, challenges, trends and future prospects. IEEE Commun Surv Tutorials 24(4):2255–2291. https://doi.org/10.1109/comst.2022.3208773

Mir M, Yaghoobi M, Khairabadi M (2022) A new approach to energy-aware routing in the Internet of Things using improved Grasshopper Metaheuristic Algorithm with Chaos theory and Fuzzy Logic. Multimed Tools Appl 82(4):5133–5159. https://doi.org/10.1007/s11042-021-11841-9

Moosavi J, Naeni LM, Fathollahi-Fard AM, Fiore U (2021) Blockchain in supply chain management: a review, bibliometric, and network analysis. Environ Sci Pollut Res. https://doi.org/10.1007/s11356-021-13094-3

Mora L, Deakin M, Reid A (2019) Combining co-citation clustering and text-based analysis to reveal the main development paths of smart cities. Technol Forecast Soc Chang 142:56–69. https://doi.org/10.1016/j.techfore.2018.07.019

Mora L, Appio FP, Foss NJ, Arellano-Gault D, Zhang X (2023a) Organizing for smart city development: research at the crossroads. Introduction to the special issue. Organ Stud 44(10):1559–1575. https://doi.org/10.1177/01708406231197815

Mora L, Gerli P, Ardito L, Messeni Petruzzelli A (2023b) Smart city governance from an innovation management perspective: theoretical framing, review of current practices, and future research agenda. Technovation 123:102717. https://doi.org/10.1016/j.technovation.2023.102717

Moya B, Badías A, Alfaro I, Chinesta F, Cueto E (2020a) Digital twins that learn and correct themselves. Int J Numer Methods Eng 123(13):3034–3044. https://doi.org/10.1002/nme.6535 . (Portico)

Moya B, Alfaro I, Gonzalez D, Chinesta F, Cueto E (2020b) Physically sound, self-learning digital twins for sloshing fluids. PLoS ONE 15(6):e0234569. https://doi.org/10.1371/journal.pone.0234569

Nämerforslund T (2022) Digital twin performance: unity as a platform for visualizing interactive digital twins. Thesis - Institution of Information Systems and Technology

National Aeronautics and Space Administration (NASA) (2010) Encyclopedia of geography. https://doi.org/10.4135/9781412939591.n797

Nica E, Popescu GH, Poliak M, Kliestik T, Sabie O-M (2023) Digital twin simulation tools, spatial cognition algorithms, and multi-sensor fusion technology in sustainable urban governance networks. Mathematics 11(9):1981. https://doi.org/10.3390/math11091981

Nikolakis N, Alexopoulos K, Xanthakis E, Chryssolouris G (2018) The digital twin implementation for linking the virtual representation of human-based production tasks to their physical counterpart in the factory-floor. Int J Comput Integr Manuf 32(1):1–12. https://doi.org/10.1080/0951192x.2018.1529430

Nochta T, Wan L, Schooling JM, Parlikad AK (2020) A socio-technical perspective on urban analytics: the case of city-scale digital twins. J Urban Technol 28(1–2):263–287. https://doi.org/10.1080/10630732.2020.1798177

Nowocin JK (2017) Microgrid risk reduction for design and validation testing using controller hardware in the loop (Doctoral dissertation, Massachusetts Institute of Technology)

ODAA platform (2016) http://www.odaa.dk . Accessed 10 May 2024

Pan S, Trentesaux D, McFarlane D, Montreuil B, Ballot E, Huang GQ (2021) Digital interoperability in logistics and supply chain management: state-of-the-art and research avenues towards Physical Internet. Comput Ind 128:103435. https://doi.org/10.1016/j.compind.2021.103435

Pang J, Huang Y, Xie Z, Li J, Cai Z (2021) Collaborative city digital twin for the COVID-19 pandemic: a federated learning solution. Tsinghua Sci Technol 26(5):759–771. https://doi.org/10.26599/tst.2021.9010026

Paripooranan CS, Abishek R, Vivek DC, Karthik S (2020) An implementation of AR enabled Digital Twins for 3-D printing. 2020 IEEE International Symposium on Smart Electronic Systems (ISES) (Formerly INiS). https://doi.org/10.1109/ises50453.2020.00043

Park H-A, Byeon G, Son W, Jo H-C, Kim J, Kim S (2020) Digital Twin for operation of microgrid: optimal scheduling in virtual space of Digital Twin. Energies 13(20):5504. https://doi.org/10.3390/en13205504

Pérez JG, Toledo DG (2017) Cognitive analytics of smart cities. Proceedings of the 18th annual international conference on digital government research. https://doi.org/10.1145/3085228.3085265

Petković T, Puljiz D, Marković I, Hein B (2019) Human intention estimation based on hidden Markov model motion validation for safe flexible robotized warehouses. Robot Comput-Integr Manuf 57:182–196. https://doi.org/10.1016/j.rcim.2018.11.004

Pivano L, Nguyen DT, Bruun Ludvigsen K (2019) Digital Twin for drilling operations – towards cloud-based operational planning. Day 3 Wed, May 08, 2019. https://doi.org/10.4043/29316-ms

Podvalny SL, Vasiljev EM (2021) Digital twin for smart electricity distribution networks. IOP Conf Ser: Mater Sci Eng 1035(1):012047. https://doi.org/10.1088/1757-899x/1035/1/012047

Prasad D, Alizadeh T, Dowling R (2023) Smart city planning and the challenges of informality in India. Dialogues in human geography, 204382062311566. https://doi.org/10.1177/20438206231156655

Praschl C, Krauss O (2022) Geo-referenced occlusion models for mixed reality applications using the microsoft HoloLens. Proceedings of the 17th international joint conference on computer vision, imaging and computer graphics theory and applications. https://doi.org/10.5220/0010775200003124

Psarommatis F, May G (2022) A standardized approach for measuring the performance and flexibility of digital twins. Int J Prod Res 61(20):6923–6938. https://doi.org/10.1080/00207543.2022.2139005

Qiuchen Lu V, Parlikad AK, Woodall P, Ranasinghe GD, Heaton J (2019) Developing a dynamic digital twin at a building level: using Cambridge campus as case study. International Conference on Smart Infrastructure and Construction 2019 (ICSIC). https://doi.org/10.1680/icsic.64669.067

Quek HY, Sielker F, Akroyd J, Bhave AN, von Richthofen A, Herthogs P, van der Laag Yamu C, Wan L, Nochta T, Burgess G, Lim MQ, Mosbach S, Kraft M (2023) The conundrum in smart city governance: interoperability and compatibility in an ever-growing ecosystem of digital twins. Data Policy 5. https://doi.org/10.1017/dap.2023.1

Rabah S, Assila A, Khouri E, Maier F, Ababsa F, Bourny V, Maier P, Mérienne F (2018) Towards improving the future of manufacturing through digital twin and augmented reality technologies. Procedia Manuf 17:460–467. https://doi.org/10.1016/j.promfg.2018.10.070

Rajesh PK, Manikandan N, Ramshankar CS, Vishwanathan T, Sathishkumar C (2019) Digital Twin of an automotive brake pad for predictive maintenance. Procedia Comput Sci 165:18–24. https://doi.org/10.1016/j.procs.2020.01.061

Raqeeb A, Bonetti A, Carlsson A, Harispuru C, Pustejovsky M, Wetterstrand N (2022) Functional digital twins of relay protection and relay test equipment enabling benefits in training and remote support. 16th International Conference on Developments in Power System Protection (DPSP 2022). https://doi.org/10.1049/icp.2022.0925

Rathore MM, Shah SA, Shukla D, Bentafat E, Bakiras S (2021) The role of AI, machine learning, and big data in digital twinning: a systematic literature review, challenges, and opportunities. IEEE Access 9:32030–32052. https://doi.org/10.1109/access.2021.3060863

Rebmann A, Knoch S, Emrich A, Fettke P, Loos P (2020) A multi-sensor approach for Digital Twins of manual assembly and commissioning. Procedia Manuf 51:549–556. https://doi.org/10.1016/j.promfg.2020.10.077

Revetria R, Tonelli F, Damiani L, Demartini M, Bisio F, Peruzzo N (2019) A real-time mechanical structures monitoring system based on Digital Twin, Iot and augmented reality. 2019 Spring Simulation Conference (SpringSim). https://doi.org/10.23919/springsim.2019.8732917

Rosen R, von Wichert G, Lo G, Bettenhausen KD (2015) About the importance of autonomy and Digital Twins for the future of manufacturing. IFAC-PapersOnLine 48(3):567–572. https://doi.org/10.1016/j.ifacol.2015.06.141

Rožanec JM, Lu J, Rupnik J, Škrjanc M, Mladenić D, Fortuna B, Zheng X, Kiritsis D (2022) Actionable cognitive twins for decision making in manufacturing. Int J Prod Res 60(2):452–478. https://doi.org/10.1080/00207543.2021.2002967

Saeed Z, Mancini F, Glusac T, Izadpanahi P (2022) Future city, digital twinning and the urban realm: a systematic literature review. Buildings 12(5):685. https://doi.org/10.3390/buildings12050685

San Francisco Open Data (2024) San Francisco Open Data. https://data.sfgov.org . Accessed 10 May 2024

Shahat E, Hyun CT, Yeom C (2021) City Digital Twin potentials: a review and research agenda. Sustainability 13(6):3386. https://doi.org/10.3390/su13063386

Shangguan D, Chen L, Ding J (2020) A Digital Twin-based approach for the fault diagnosis and health monitoring of a complex satellite system. Symmetry 12(8):1307. https://doi.org/10.3390/sym12081307

Srai J, Settanni E (2019) Supply chain digital twins: opportunities and challenges beyond the hype

Ssin S, Cho H, Woo W (2021a) GeoACT: augmented control tower using virtual and real geospatial data. Interact Des Archit 48:122–142. https://doi.org/10.55612/s-5002-048-006

Ssin S, Cho H, Woo W (2021b) GeoVCM: virtual urban digital twin system augmenting virtual and real geo-spacial data. 2021 IEEE International Conference on Consumer Electronics (ICCE). https://doi.org/10.1109/icce50685.2021.9427709

Stark R, Kind S, Neumeyer S (2017) Innovations in digital modelling for next generation manufacturing system design. CIRP Ann 66(1):169–172. https://doi.org/10.1016/j.cirp.2017.04.045

Steinmetz C, Schroeder GN, Sulak A, Tuna K, Binotto A, Rettberg A, Pereira CE (2022) A methodology for creating semantic digital twin models supported by knowledge graphs. 2022 IEEE 27th International Conference on Emerging Technologies and Factory Automation (ETFA). https://doi.org/10.1109/etfa52439.2022.9921499

Svítek M, Skobelev P, Kozhevnikov S (2019) Smart city 5.0 as an urban ecosystem of smart services. Stud Comp Intell:426–438. https://doi.org/10.1007/978-3-030-27477-1_33

Tao F, Cheng J, Qi Q, Zhang M, Zhang H, Sui F (2017) Digital twin-driven product design, manufacturing and service with big data. Int J Adv Manuf Technol 94(9–12):3563–3576. https://doi.org/10.1007/s00170-017-0233-1

Ton DT, Smith MA (2012) The U.S. Department of Energy’s Microgrid Initiative. Electr J 25(8):84–94. https://doi.org/10.1016/j.tej.2012.09.013

Tonnarelli F, Mora L (2023) Smart urbanism in Africa: when theories do not fit with contextual practices. Reg Stud:1–11. https://doi.org/10.1080/00343404.2023.2235407

Trusov A, Limonova EE (2020) The analysis of projective transformation algorithms for image recognition on mobile devices. Twelfth International Conference on Machine Vision (ICMV 2019). https://doi.org/10.1117/12.2559732

Tuhaise VV, Tah JHM, Abanda FH (2023) Technologies for digital twin applications in construction. Autom Constr 152:104931. https://doi.org/10.1016/j.autcon.2023.104931

Tygesen UT, Jepsen MS, Vestermark J, Dollerup N, Pedersen A (2018) The true digital twin concept for fatigue re-assessment of marine structures. Volume 1: offshore technology. https://doi.org/10.1115/omae2018-77915

Um J, Popper J, Ruskowski M (2018) Modular augmented reality platform for smart operator in production environment. 2018 IEEE Industrial Cyber-Physical Systems (ICPS). https://doi.org/10.1109/icphys.2018.8390796

Urbina Coronado PD, Lynn R, Louhichi W, Parto M, Wescoat E, Kurfess T (2018) Part data integration in the shop floor digital twin: mobile and cloud technologies to enable a manufacturing execution system. J Manuf Syst 48:25–33. https://doi.org/10.1016/j.jmsy.2018.02.002

van der Valk H, Strobel G, Winkelmann S, Hunker J, Tomczyk M (2022) Supply chains in the era of Digital Twins – a review. Procedia Comput Sci 204:156–163. https://doi.org/10.1016/j.procs.2022.08.019

Verdouw C, Tekinerdogan B, Beulens A, Wolfert S (2021) Digital twins in smart farming. Agric Syst 189:103046. https://doi.org/10.1016/j.agsy.2020.103046

Vidal-Balea A, Blanco-Novoa O, Fraga-Lamas P, Vilar-Montesinos M, Fernández-Caramés TM (2021) Collaborative augmented digital twin: a novel open-source augmented reality solution for training and maintenance processes in the shipyard of the future. The 4th XoveTIC conference. https://doi.org/10.3390/engproc2021007010

Wan L, Nochta T, Schooling JM (2019) Developing a city-level digital twin –propositions and a case study. International Conference on Smart Infrastructure and Construction 2019 (ICSIC). https://doi.org/10.1680/icsic.64669.187

Wang P, Luo M (2021) A digital twin-based big data virtual and real fusion learning reference framework supported by industrial internet towards smart manufacturing. J Manuf Syst 58:16–32. https://doi.org/10.1016/j.jmsy.2020.11.012

Wang W, He F, Li Y, Tang S, Li X, Xia J, Lv Z (2023) Data information processing of traffic digital twins in smart cities using edge intelligent federation learning. Inf Process Manage 60(2):103171. https://doi.org/10.1016/j.ipm.2022.103171

Wang H, Meng X (2021) BIM-supported knowledge management: potentials and expectations. J Manag Eng 37(4). https://doi.org/10.1061/(asce)me.1943-5479.0000934

Waszak M, Lam AN, Hoffmann V, Elvesater B, Mogos MF, Roman D (2022) Let the asset decide: digital twins with knowledge graphs. 2022 IEEE 19th International Conference on Software Architecture Companion (ICSA-C). https://doi.org/10.1109/icsa-c54293.2022.00014

Wen T, Dobson E, Hvaara R (2020) Mesh learning: a cloud and edge–based computing network providing data–driven solutions to the oil and gas industry. Day 2 Tue, November 03, 2020. https://doi.org/10.4043/30365-ms

White G, Zink A, Codecá L, Clarke S (2021) A digital twin smart city for citizen feedback. Cities 110:103064. https://doi.org/10.1016/j.cities.2020.103064

Wiegand G, Mai C, Liu Y, Hußmann H (2018) Early take-over preparation in stereoscopic 3D. Adjunct proceedings of the 10th international conference on automotive user interfaces and interactive vehicular applications. https://doi.org/10.1145/3239092.3265957

Xia L, Zheng P, Li X, Gao RX, Wang L (2022) Toward cognitive predictive maintenance: a survey of graph-based approaches. J Manuf Syst 64:107–120. https://doi.org/10.1016/j.jmsy.2022.06.002

Xie X, Lu Q, Rodenas-Herraiz D, Parlikad AK, Schooling JM (2020) Visualised inspection system for monitoring environmental anomalies during daily operation and maintenance. Eng Constr Archit Manag 27(8):1835–1852. https://doi.org/10.1108/ecam-11-2019-0640

Xu Y, Sun Y, Liu X, Zheng Y (2019) A Digital-Twin-assisted fault diagnosis using deep transfer learning. IEEE Access 7:19990–19999. https://doi.org/10.1109/access.2018.2890566

Xu W, Cui J, Li L, Yao B, Tian S, Zhou Z (2021) Digital twin-based industrial cloud robotics: framework, control approach and implementation. J Manuf Syst 58:196–209. https://doi.org/10.1016/j.jmsy.2020.07.013

Xu H, Berres A, Yoginath SB, Sorensen H, Nugent PJ, Severino J, Tennille SA, Moore A, Jones W, Sanyal J (2023) Smart mobility in the cloud: enabling real-time situational awareness and cyber-physical control through a Digital Twin for traffic. IEEE Trans Intell Transp Syst 24(3):3145–3156. https://doi.org/10.1109/tits.2022.3226746

Yan M, Gan W, Zhou Y, Wen J, Yao W (2022) Projection method for blockchain-enabled non-iterative decentralized management in integrated natural gas-electric systems and its application in digital twin modelling. Appl Energy 311:118645. https://doi.org/10.1016/j.apenergy.2022.118645

Yin Y, Zheng P, Li C, Wang L (2023) A state-of-the-art survey on augmented reality-assisted Digital Twin for futuristic human-centric industry transformation. Robot Comput-Integr Manuf 81:102515. https://doi.org/10.1016/j.rcim.2022.102515

Yu G, Wang Y, Hu M, Shi L, Mao Z, Sugumaran V (2021) RIOMS: an intelligent system for operation and maintenance of urban roads using spatio-temporal data in smart cities. Futur Gener Comput Syst 115:583–609. https://doi.org/10.1016/j.future.2020.09.010

Yu X, Merritt J (2023) Comparison of city digital twin case studies. Digital twins for smart cities, 123–137. https://doi.org/10.1680/dtsc.66007.123

Zakzak L (2019) Citizen-centric smart city development. Proceedings of the 20th annual international conference on digital government research. https://doi.org/10.1145/3325112.3325236

Zborowski M (2018) Finding meaning, application for the much-discussed “Digital Twin.” J Petrol Technol 70(06):26–32. https://doi.org/10.2118/0618-0026-jpt

Zhang J, Mora L (2023) Nothing but symbolic: Chinese new authoritarianism, smart government, and the challenge of multi-level governance. Gov Inf Q 40(4):101880. https://doi.org/10.1016/j.giq.2023.101880

Zhang K, Cao J, Zhang Y (2022) Adaptive digital twin and multiagent deep reinforcement learning for vehicular edge computing and networks. IEEE Trans Industr Inf 18(2):1405–1413. https://doi.org/10.1109/tii.2021.3088407

Zhao X, Sun Y (2020) Augmented reality assembly guidance method based on situation awareness. In: Proceedings of the 2020 the 10th international workshop on computer science and engineering. WCSE, pp 165–173

Zheng P, Lin T-J, Chen C-H, Xu X (2018) A systematic design approach for service innovation of smart product-service systems. J Clean Prod 201:657–667. https://doi.org/10.1016/j.jclepro.2018.08.101

Zheng X, Lu J, Kiritsis D (2021) The emergence of cognitive digital twin: vision, challenges and opportunities. Int J Prod Res 60(24):7610–7632. https://doi.org/10.1080/00207543.2021.2014591

Zheng P, Li S, Fan J, Li C, Wang L (2023) A collaborative intelligence-based approach for handling human-robot collaboration uncertainties. CIRP Ann 72(1):1–4. https://doi.org/10.1016/j.cirp.2023.04.057

Židek K, Hladký V, Pitel’ J, Demčák J, Hošovský A, Lazorík P (2021) SMART production system with full digitalization for assembly and inspection in concept of industry 4.0. Future access enablers for ubiquitous and intelligent infrastructures. pp 181–192. https://doi.org/10.1007/978-3-030-78459-1_13

Zohdi TI (2020) A machine-learning framework for rapid adaptive digital-twin based fire-propagation simulation in complex environments. Comput Methods Appl Mech Eng 363:112907. https://doi.org/10.1016/j.cma.2020.112907

Article   MathSciNet   Google Scholar  

Download references

Acknowledgements

The authors extend their appreciation to the Deputyship for Research and Innovation, Ministry of Education in Saudi Arabia, for funding this research work through project number 445-5-961.

This research was funded by the Deputyship for Research and Innovation, Ministry of Education in Saudi Arabia, project number 445-5-961.

Author information

Authors and affiliations.

College of Computer Science and Engineering, Taibah University, 46421, Yanbu, Saudi Arabia

Rasha F. El-Agamy, Hanaa A. Sayed, Arwa M. AL Akhatatneh, Mansourah Aljohani & Mostafa Elhosseini

Computer Science Department, Faculty of Science, Tanta University, Tanta, 31527, Egypt

Rasha F. El-Agamy

Department of Computer Science, Faculty of Computers and Information, Assiut University, Assiut, 71516, Egypt

Hanaa A. Sayed

Department of Computers and Control Systems Engineering, Faculty of Engineering, Mansoura University, Mansoura, 35516, Egypt

Mostafa Elhosseini

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization, M.E.; methodology, R.E., H.A., A.A., M.A.; software, A.A. and M.A.; validation, H.A., R.E. and M.E.; formal analysis, R.E.. and A.A.; investigation, A.A. and H.A; resources, M.A.; data curation, R.E., and H.A; writing—original draft preparation, R.E., H.A., A.A., and M.A.; writing—review and editing, R.E., H.A., and M.E.; visualization, R.E., and H.A; supervision, M.E., H.A., and R.E.; project administration, M.E.; funding acquisition, M.E. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Mostafa Elhosseini .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

El-Agamy, R.F., Sayed, H.A., AL Akhatatneh, A.M. et al. Comprehensive analysis of digital twins in smart cities: a 4200-paper bibliometric study. Artif Intell Rev 57 , 154 (2024). https://doi.org/10.1007/s10462-024-10781-8

Download citation

Accepted : 29 April 2024

Published : 27 May 2024

DOI : https://doi.org/10.1007/s10462-024-10781-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Digital twin
• Smart city development
• Bibliometric study
• IoT integration
• Find a journal
• Publish with us
• Track your research