Evaluation of the e-Learning Utilization as a Learning System in Higher Education Institution

— This study aims to evaluate the use of technology to support teaching and learning activities. Lecturers and students have applied e-learning to teach subjects. The purpose of this evaluation is to measure the success of the use of STMIK Bumigora e-learning by using the Technology Acceptance Model (TAM) approach, which is an approach that can explain user behavior towards the use of technology. Evaluation of the use of e-learning is formulated into a model based on the TAM model, while SEM (Structural Equation Modelling) is used for data analysis. Based on the measurement analysis in this study, several factors most influenced the effectiveness of e-learning, namely the usage tutorial for users, ICT facilities related to the Ease of accessing the internet network. Meanwhile, in structural analysis, it was found that attitudes toward the use and perceived usefulness were strongly correlated with real use factors. The actual use is a real condition of the use of e-learning measured by the frequency and duration of time in using the technology, which is influenced by the user's belief in accepting the existence of e-learning in STMIK Bumigora and user beliefs related to the benefits when using it. Therefore, attitudes toward the use and perception of usefulness are the main determining factors in measuring the frequency and duration of e-learning use.


INTRODUCTION
E-learning or online learning has been developed by the Information and communications technology (ICT) center of STMIK Bumigora Mataram to support the learning process activity. This application is one of the learning models chosen by students to manage their course activities flexibly. This application is carried out based on e-learning as a supplement, complement, and substitute [1]. Mean, students can use e-learning to increase their knowledge and insight [2] through teaching material that has been prepared as enrichment material for students who take conventional learning or face-toface.
Besides, as the learning model, e-learning usage is one of the strategies and support in strategic planning of information systems or information technology by the objectives of STMIK Bumigora. Therefore, online learning is one of the mappings of information systems applications to achieve the vision and mission goals at STMIK Bumigora [3]. E-learning, which had been developed, can be accessed from students, lecturers, and the admin side. It is equipped with several features such as 'how to access' features for students, including a signup button, log in, join the group, and upload an assignment. Besides, there is a tutorial on how to access e-learning for lecturers and admin. The lecture site contains how to log in, create a group, add course material and assignments on e-learning, download assignments, edit profiles, change passwords, and enroll students into the course. There are features such as a login button, back up lecturer group 161 Jurnal Infotel Vol. 12 No. 4 November 2020 https://doi.org/10.20895/infotel.v12i4.475 discussion, edit profile, user data, role permissions, and change password for the admin site. This elearning system was developed with Learning Management System (LMS) for its learning process and class management, creating course material and content, discussion forums, grading system, examination system, online quiz, creating video, etc.
As a learning development tool program, LMS was known in 1990 related to the management of education in tertiary institutions [4]. On this basis, lectures' content c in the form of modules, text, pictures, tables, etc., can be accessed by students so that the lecturer can control the student's development online as report material. The representation of elearning is shown in Fig.1.  Figure 1 shows the interface of the main page of STMIK Bumigora's e-learning system. It provides several features for learning process management, containing the groups based on the lecturer and their course materials. The group feature contains course material such as interactive multimedia or text, which is stored in LMS. It is easy to access by students flexibly, based on their location and time.
To ensure the implementation of e-learning, supporting facilities such as a server, computer client, computer network, and internet access are already prepared. Therefore, the learning component, such as system and e-learning applications like LMS to manage the online course learning, had been fulfilled. The course material that can be displayed may consist of mobile content and text that students can access and the conditions of on-hand ICT facilities such as infrastructure consisting of client PCs and internet networks.. Asynchronous learning is the learning method applied in the e-learning system. This method means that the instructor and the student are in the same class as a group, even at different times and places. Thus the role of the LMS-based e-learning application system is needed. It contains teaching material that can be accessed for 24 hours through the internet network.
The lecturer and students can access e-learning facilities as the user. It can increase the student's knowledge and skills through ICT utilization [3]. Hence, if the instructor could not deliver the course materials or do evaluation directly, e-learning could solve them online.
The availability of e-learning facilities is contrary to the frequency of the number of users at the institution. This information is based on the data recapitulation in the e-learning application system. Information from the ICT center related to its utilization is minimal, and no evaluation has been carried out. Based on data in 2016, with 60 lecturers and 1023 students, only five lecturers and 500 students have utilized e-learning.
Therefore, a study of the factors that most contributed to the effectiveness of e-learning at STMIK Bumigora was conducted. This study is done as one of the evaluation materials for using e-learning information systems at the institution. The results of this evaluation serve as guidelines in efforts to improve and develop e-learning applications.
This study uses the Technology Accepted Model (TAM) approach that Fred Davis has developed to measure user behavior towards technology use. TAM is one of the most widely used approaches in information systems research related to one's perception of the Ease of utilizing technology. [5] [2]. While the data analysis used is with structural Equation Modelling (SEM). SEM is referred to as a mathematical and statistical model [6][7], path analysis [8] to measure variables that cannot be observed [9] so that it requires observable indicators or variables. In the path analysis, it is said that all variables form a regression equation like the one in LISREL [10], AMOS and SmartPLS However, various researchers have criticized this method throughout its development regarding a limited theory's predictive power and changes when it is associated with information technology [11]. Therefore, Pelin's evaluation analysis uses TAM by paying attention to how individual users can feel using the technology. This method ignores the social essence and assessment of the use of better technology [12] Or in other words, the TAM method is only a small part of the use of a technological system [13]. This method has been used by researchers related to UKM in Malaysia and Hong Kong., The method found that the adoption of online multimedia technology was not influenced by the Ease of use but was determined by participant's willingness to learn online multimedia technology. Many studies have been carried out related to structural equation models, namely, among others in science, such as testing Oxygen's concentration (Dissolved Oxygen) in determining river water quality related to pollutants and sedimentation. Oxygen levels affect the level of river pollution or pollutants. Oxygen can be associated with sedimentation. DO is a variable affected by sedimentation. The more sedimentation in the river, the level of oxygen decomposition decreases [14]. Research in other fields, such as the social field, measures human intelligence, which is a variable that cannot be measured directly. Therefore, indicators such as instruments and measuring intelligence are needed. For example, an indicator of human intelligence when viewed from an academic performance such as the SAT, ACT, and school GPA. From each indicator of academic performance analyzed using SEM will be obtained one of the most influential indicators according to existing theories [8].

II. RESEARCH METHOD
The research method in this study can be described in Fig.2.  Figure 2 shows the preparation stage, the data collection, the data analysis, and the completion stage. The details are as follows: a. The preparation stage, related to literature review related to e-learning evaluation such as analytical methods, indicators of measuring variables for e-learning evaluation using TAM b. The data collection stage, the TAM model is formulated with statistical analysis, namely SEM, based on the theory relevant to the problem to be studied. Modeling also depends on the sample data taken. In this case, random sampling will be carried out to determine the object of research, and then the questionnaire will be distributed to be filled out by lecturers and students c. The data analysis stage, a simulation of the data is carried out and adjusted to the model that has been compiled, namely the TAM model with SEM statistics. Before the simulation, first, the data validation test was carried out related to normality and multicollinearity. At this stage, it can also be seen whether the data and model are appropriate so that there is an evaluation of whether the data or model is changed so that it is valid to draw conclusions d. The completion stage, from the results of the interpretation of the results, will formulate several conclusions that higher education leaders can use in measuring the application of e-learning

A. Establishment of e-Learning Evaluation
The e-learning acceptance model will be analyzed using a structural equation model called SEM. The model is formed based on several measuring indicators, as shown in Table 1 [2].

B. E-learning Acceptance Model Formulation
Each exogenous and endogenous latent variable has three measuring indicators and is analyzed based on structural equation modeling procedures, Model Specifications. At this stage, it is building hypotheses based on problems found in the field, arranging the variables that are modeled, and defining the structural relationships between variables, as shown in Fig.3. Analysis of the evaluation of e-learning, called Pelin, is analyzed using SEM based on a TAM model. Several assumptions must be met in the analysis test phase using SEM.

a. Classical Test Assumptions on Sample Data
Analysis of classical testing is used to determine the validity of sample data to achieve this study's objectives. The classic test consists of several analyzes, namely: • Descriptive Analysis of Indicator Variables A total of 130 data were used as samples, consisting of maximum, minimum, average, and standard deviation of the measuring indicator variables.

• Data Normality Test
The value of Skewness and Kurtosis is seen not only based on the degree of asymmetry of distribution but also the degree of fineness of distribution, respectively. The analysis results concluded that the sample data derived from the measuring indicator variables is normal for further analysis research.

• Multicollinearity Test
This test examines the disconnection between the measuring indicator variables and the results obtained that do not work multicollinear.
b. Confirmatory Factor Analysis/CFA Confirmatory factor analysis is performed to see a constructed or observed variable or measure's validity and reliability. Therefore, in this case, there are six construct variables, which are three indicators measuring Training (PLT) and three indicators measuring the Condition of Technology Facilities (KFT). AMOS has modified the path diagram to illustrate the relationship between indicator variables. The analysis of the confirmatory factor shows in Table 2.  Table 2 shows that the analysis of confirmatory factors using SEM obtained all valid measurement variables if observed from the loading factor criteria value is greater than 0.5 and significant to measure endogenous latent variables.   Table 3 shows that there is 11 goodness of fit criteria producing 7 Fit or Good evaluation models so that it can be concluded that the exogenous variable model is Good. However, an index modification is needed for the exogenous variables, which can be shown in Table 4. The testing of models for endogenous variables can be shown in Table 5.  Table 5 shows the results of testing endogenous variables are Poor so that it needs a modification of the Index and based on the analysis using AMOS to find the relationship between indicators on endogenous variables so that the test results are shown in Table 6. Based on Table 6, it can conclude that the testing of models for endogenous variables is Good overall, although there are two criteria for poor model compatibility.
c. The structural model equation aims to analyze the relationship between latent variables.
Structural model equations describe the relationships between exogenous and endogenous latent variables, and the overall model fit is tested, as shown in Fig. 4. The path diagram in Fig.4 shows the relationship between exogenous variables and their indicators, endogenous variables with their indicators, and the relationship between latent variables. Before estimating, a comprehensive model testing is needed and can be tabulated as Table 7. The results of the overall model suitability test, as shown in Table 7, are Poor. As in testing the model for exogenous and endogenous, so modifications are needed to obtain better values. The Index's modification aims to find the relationship that occurs between parameters and the results of the modification of the Index in the overall model testing, namely, The modification of the Index for the whole model's suitability is shown in Table 8. It can conclude that the e-learning usage model at STMIK Bumigora Mataram is suitable or good. Therefore, the interpretation analysis can make the right decision according to the problems that occur.

IV. DISCUSSION
The e-learning utilization is evaluated using the Technology Acceptance Model (TAM) model to determine the use of technology among users, as has been done by previous researchers. Several things need to be considered in the formation of models using TAM, namely the perception of benefits and perceived Ease of use of the technology as formulated by Davis in his theory using The Theory of Reasoned Action (TRA) [15] [16] as shown in Fig.5.   [2]. These external variables affect the acceptance of technology users. For example, training indicators aim to increase confidence in using technology. Also, KFT is used as an internet network support, Motivation (MOT) related to the use of new technologies [17], and KDI related to user confidence in technology.
The Perceived Ease of use variable is also referred to as perceived Ease (MDH), meaning that it is easy to learn, use, and access users [18]. Perception of usefulness as perceived usefulness related to the benefits obtained by users. For example, users feel helped incomplete tasks, more effective and efficient in learning, and more productive. The attitude of Use (SKP) is called attitude toward use because it gives effect to the user [13], and the real use variable is called Actual Use, which is the result of testing and prediction of whether the user accepts or rejects new information technology [19] [20]. Based on this description, the TAM model's path analysis (Fig.5) can be developed according to the variables and indicators, as shown in Fig.3. The path analysis shows the relationship of each latent variable and its measuring indicators mentioned in Table 1 The use of e-learning in STMIK Bumigora Mataram will be evaluated related to the problem of quantity or number of e-learning users far from expectations. A model is formed based on the Technology Acceptance Model (TAM) [21]. Analysis using Structural Equation Model (SEM) with the help of AMOS. Model testing has been formed, and the model's suitability is tested to obtain a better model. The estimation results will be used for the interpretation of results, namely:

A. Estimation of the Measurement Model Equation
The following Table 9 presents a measurement model equation consists of exogenous and endogenous variables with their measuring indicators. Based on Table 9, the most significant indicators for exogenous latent variables are indicators X3 and X4, respectively. X3 is the Training (PLT), which states the effectiveness in implementation, while X4 is the Condition of ICT Facilities (KFT), which states the Ease of getting internet access. Whereas for endogenous variables, the most significant indicators are Y1, Y2, and Y3 on latent variables Usage Motivation (MOT), Y6 on Confidence using the Internet (KDI), Y7 on Perception of Ease (MDH), Y10, Y11, and Y12 on Perception of Usability (GUNA), Y16, Y17, and Y18 on Real Use Attitudes (NYATA). The purpose of estimating the measurement model equation is to discover how significant is the relationship between exogenous or endogenous latent variables and their constituent indicators.

B. Estimation of Structural Equation Models
The magnitude of the relationship between latent variables can be tabulated based on the estimation results analyzed using SEM, namely:   Table 10 shows that the loading factor value is greater than 0.5, so it can conclude that all the structural model parameters are significant. In other words, the structural model shows that each parameter is correlated. For example, Training (PLT) has a positive correlation with KDI of 0.535, meaning that 53.5% of the variation in PLT can explain confidence using the Internet (KDI) variations, namely if someone is motivated to use technology effectively, that person will be accustomed to activities with technology. Likewise, other parameters have different correlation values. Use Perception (GUNA) and Usage Approval (SKP) have the most significant positive relationship with Real Use (NYATA), which means that technology developed such as e-learning is accepted. The acceptance due to the user is perceived benefits, such as completing a task, users more effectively and efficiently in the teaching and learning process. It is used to influence aspects of the user in using the technology significantly.

V. CONCLUSION
New technology training activities for users are needed to achieve effectiveness in their application accompanied by adequate ICT facilities, namely the Ease of getting internet access. Also, legal regulations from institutions, funding support, and Perceived Ease of technology use such as e-learning are the most important things in building user's motivation, both lecturers and students. The habit of using the Internet for more than 3 hours is a form of confidence in using the Internet because it is easy to learn, and users have perceptions of uses in using e-learning such as feeling helped in completing tasks, more effective in teaching and learning process, and increasing in productivity. This condition is a reflective material for making decisions regarding the acceptance of e-learning in STMIK Bumigora Mataram in its actual use, such as users consistently using applications, only using a small part and specific functions. This description is an evaluation material for STMIK Bumigora Mataram. Several things must be evaluated, namely the lack of training and inadequate conditions of ICT facilities so that users do not have confidence in using the Internet and users cannot feel the benefits of elearning that has been developed. Therefore, efforts are needed to overcome this.
Perception of usability and attitude towards use have the most significant positive relationship with real use, meaning that technology developed such as e-learning is accepted or not because of the user's perceived benefits such as completing assignments, effectiveness, and teaching efficiency learning process. This activity is used as an aspect to influence the user in using technology significantly.
There are two exogenous variables used in developing an evaluation model for the e-learning utilization as a lecture system at STMIK Bumigora Mataram, namely the Training and Condition of ICT Facilities. Therefore, it is recommended to further researchers add several exogenous variables to improve the results of this study.