Assessment decisions of independent learning activities using SMART–FCM method

— Sekolah Tinggi Ilmu Komputer (STIKOM) PGRI Banyuwangi implemented the Independent Learning - Independent Campus (MBKM) activity for two semesters. The results of student assessments for MBKM activities for one semester are influenced by the results of daily and weekly logbook monitoring, monitoring and evaluation assessments, and assessments of supervisors, examiners, and work partners. Assessments that are less objective cause many students to get good grades even though the implementation of MBKM activities is not well. The Simple Multi-Attribute Rating Technique (SMART) method is used to produce student eligibility group data and a more objective assessment. The results of the SMART calculations are combined with the Fuzzy C − Means (FCM) algorithm so that the results of grouping student data are more appropriate based on the similarities and characteristics of the members. To find the best data grouping results between the SMART and SMART − FCM methods, the Silhouette Coefficient is used to compare the grouping results. The results obtained that the use of SMART − FCM is better than the SMART results because it has a Silhouette Coefficient value close to 1 of 0.31187. This proves that the decision results using SMART will be more accurate using the FCM method. and the results of the decision can be used as a reference in assessing student MBKM activities.


I. INTRODUCTION
Sekolah Tinggi Ilmu Komputer (STIKOM) PGRI Banyuwangi has implemented the Independent Learning -Independence Campus (MBKM) activity based on the policies issued by the Ministry of Education and Culture Republic of Indonesia, as stipulated in Permendikbud No. 3 in 2020 [1], by taking three forms of activities in learning outside of tertiary institutions for two semesters, including internships, independent projects, and entrepreneurship from a total of eight activities [2].
In implementing learning outside of tertiary institutions, MBKM activities have different regulations from regular lectures in general, starting from academic selection, registration, and implementation to the end of the activities contained in the form of student accountability reports based on all activities that have been conducted. Then the accountability session was held to assess student activities' results for one semester based on the results of daily and weekly logbook monitoring, monitoring and evaluation assessments, and assessments of supervisors, examiners, and work partners. With this strict policy process, students are expected to be ready to face the challenges of the industrial revolution 4.0 [3].
The problems found were the lack of objective assessments conducted by supervisors and examiners and the weight of the assessments was not balanced causing many students to get good grades even though the implementation of MBKM activities was not so good. A Decision Support System (DSS) is used for student assessment and student grouping in determining eligible, considered, and ineligible student groups. A decision support system is a part of a computerbased information system (by implementing mathematical and analytical modeling, database information, Jurnal Infotel, Vol. 15 and user interfaces) that is used to support a decision system in an organization or company [4], [5]. There are several methods that can be used in decision systems. One of them is the Simple Multi-Attribute Rating Technique (SMART), where this method is a multi-criteria decision method in which each criterion has a weighted value and an alternative that shows how important the criteria are in making decisions with the simplest solutions [6], [7].
To assist in the effective and efficient grouping of student data based on the results of decisions obtained from the SMART method, it is necessary to cluster student data. Clustering is an unsupervised technique based on analysis and data mining techniques [4], where the process will group members based on member equations on each partition in a certain matrix. Based on cluster analysis, the goal is to group n objects or individuals into several m clusters based on their characteristics [8], so that clusters have different characteristics between groups, while each group has relatively homogeneous characteristics. In analyzing data, several processes are needed, including standardization of data, measurement of object similarity, and selection of cluster analysis procedures.
Fuzzy C−Means (FCM) is a clustering method with the concept of grouping data based on the degree of data membership. FCM is soft clustering because it is based on fuzzy logic, a data can be part of two or more clusters with membership weights ranging between 0 and 1 [9]. The FCM process calculates the distance between the center of the cluster and each data point so that the membership of all data points to each cluster center can be determined.
In previous studies, Tejawati et al. used the SMART method to find out the level of drug addicts [10], while Noviani et al. [11] used the method to achieve employee performance that affects the success of an organization. In 2020, Marlinda et al. [12] use the SMART method for the selection of Indonesian online sales factors that affect women's business loyalty. The results of these studies indicate that the SMART method can produce more structured, systematic, and transparent results with a success rate. In 2017, Asgharizadeh et al. [13] discussed the new output-oriented classification of the Multiple Attribute Decision Making (MADM) techniques, where the classification was input-oriented or process-oriented. In measuring the performance of seventeen MADM techniques for classification results based on seven performance variables, the results of each MADM technique would be clustered using FCM. The results showed that the considered techniques could be classified into two groups.
Research by Swindiarto et al. [14] integrated FCM clustering and TOPSIS in multi-criteria parameter data at PT. XYZ, where a comparison of analysis between branch offices was needed to be based on one factor against another. From the weighting by using the highest membership value in the clustering results of each criterion parameter in the ranking, it was used as a performance evaluation for all branches of the company. Using the SMART method, Siregar et al. [15] describe if this multi-criteria decision-making theory had a meaning where each alternative had criteria and had value and weight. A problem would be classified in the form of multiobjective and multicriteria. Based on the results of tests carried out with many dynamic alternatives and using three criteria, the calculation process did not require a long time, but it would require a longer processing time if the alternative was added dynamically with a constant number of alternatives.
From the several studies mentioned above, this study focuses on the decision results produced by the SMART method with FCM will have more accurate results for classifying student eligibility in MBKM activities compared to using only the SMART method. Then the results of the decisions can be used as a reference for the campus as an evaluation material for the assessment and participation of student MBKM activities in the coming semester. With the right decisions, the quality of students participating in MBKM activities can be further improved.
For a better understanding, we organize this paper as follows. Section II discusses the theoretical analysis and algorithm analysis that explain the methods used in this study and the flow order of the cluster system, respectively. The result is presented in section III, while the discussion explains the results of the research summary on the proposed system flow experiment is shown in section IV. Finally, the conclusion of this research is explained in section V.

II. RESEARCH METHOD
This section discusses the materials and methods of the research.

A. Material
The research focused on comparing the results of student data grouping decisions based on the results of the SMART and SMART-FCM methods using student data who took part in MBKM activities in the Even 2021/2022 academic year at STIKOM PGRI Banyuwangi as many as 52 student data shown in Table 1. This is data that is recorded based on the results of student assessments starting from monitoring and evaluation, logbooks, as well as assessments of examiners and partners.
52 student data will be grouped based on the level of eligibility in the recommendations for participation in the implementation of MBKM activities in the coming semester which will be divided into three groups, namely feasible, considered, and not feasible. X 1 is the assessment of monitoring and evaluation, X 2 is the logbook value, X 3 is the value of the assistant lecturer, X 4 is the value of the examiner lecturer, and X 5 is obtained from the value of the Cooperation Partner. Based on the results of grouping the two proposed methods, the best grouping results will be a reference for universities in deciding the continuation of the implementation and assessment of student MBKM activities.

B. Method
The design in the stages of this research is shown in Fig. 1 where the discussion is the preparation of data from STIKOM PGRI Banyuwangi regarding the implementation of MBKM activities. Then a system design is made for grouping student data that discusses initial data initialization for the SMART and SMART-FCM methods. SMART is a multi-criteria decision method in which each criterion has a weighted value and an alternative that shows how important the criterion is by calculating the utility value for each criterion based on the nature of the criteria itself [16]. In the analysis of determining criteria using the SMART method, there are several identification criteria, including determining criteria and alternative data, determining the weights for each criterion, then normalizing each criterion weight to obtain the following relative weights, as shown in (1) [17].
where w j is the weighted value, w j is the total number of weighted values, and W j is the relative weight values. Then calculate the evaluation factor for each alternative data with (2).
where maxP j is the maximum parameter value, A i is the result of multiplication of alternative data values with parameter weights, and u ij is the value of the evaluation factor. The last stage is to determine the evaluation weight for each alternative data using the SMART method with (3).
where i is the alternative data value, jis the parameter data, and u i is the SMART method value.
After getting the evaluation weight of each alternative data. The result of the data recapitulation is the final result of the SMART method which can be ranked from the highest to the lowest value [18]. The next step is to test MBKM activity data using the SMART and SMART-FCM methods.
The steps taken in the Fuzzy C-Means algorithm are to determine the number of clusters of 3 clusters, the smallest error expected by epsilon (ϵ) is 0.0001, the maximum iteration value, and the initial partition matrix value of each data in each cluster randomly [14]. Calculate the amount of each data in the normalization of the student dataset with the initial partition matrix data with (4) [9].
where µ is an element of the initial partition matrix, where i = 1, 2, · · · , n and k = 1, 2, · · · , c, and Q i is the sum of each column of the random values of a matrix. From the degree of membership of the three clusters, calculate the average value of each cluster to get the value of the cluster center. Then calculate the average value of the objective function between the normalization data and the cluster center with (6).
where P t is the objective function on the t-iteration.
The FCM process will stop if the value of the objective function minus the value of the objective function of the previous iteration is less than the epsilon value or the maximum iteration has been reached.
The general design of the system is shown in Fig. 2 where the description is that the first-time data is prepared for students participating in MBKM activities, then the data normalization process is carried out using the Min−Max method. Data normalization is  used to scale data values within a specified range to facilitate calculation steps such as similarity calculations or clustering operations. One of the normalization methods is the Min−Max method, which is a simple technique for scaling values based on specified limits. The Min−Max equation is shown in (7) [19].
where X is the data value, X min and X max are the smallest and largest values of all data, and X i is the result of the normalized value. The stages of the Min−Max method are carried out by finding the smallest and largest values in the dataset resulting from the assessment of MBKM activities first, by using (7), this method will produce a new dataset with a range of values from 0 to 1 [20].
The purpose of this normalization is to simplify the calculation process. Furthermore, the initialization of weighting criteria data and initial data are determined for the Fuzzy C−Means algorithm. The normalized data is then processed using the SMART method. Based on the calculation results obtained from the SMART method, then a new dataset is obtained and the student data is grouped into 3 clusters, namely feasible, considered, and not feasible.
Based on the new dataset from the SMART results, the FCM algorithm is used to group student data into 3 clusters using the SMART method. After the grouping process is completed using the SMART and SMART−FCM methods, the results of the data grouping will be validated using the Silhouette Coefficient method because it is an effective and most popular internal measure to evaluate the cluster validity [21].
Silhouette Coefficient is used to calculate the accuracy of data grouping. The formula used in the calculation of the Silhouette Coefficient is defined in (8) [22].
where |k| is the number of k clusters, while S (ci) is the average distance between object i and all objects in a cluster. The value generated by the calculation of the Silhouette Coefficient is -1 to 1 [23], [24]. The average result of S (c) for all data in a cluster shows the accuracy in grouping the data. The closer to 1, the resulting clustering structure is correct, if -1 then the resulting clustering structure is overlapping [8].
From the results of the calculation of the Silhouette Coefficient on the results of the SMART and FCM methods, they will be compared to find the best data grouping results from the two methods. The results of the evaluation of the two methods will be compared as a conclusion to determine which clustering results are the best in grouping student MBKM activity data for future decisions.

III. RESULT
From the dataset shown in Table 1, it is normalized using the Min−Max algorithm. The search for the smallest and largest values is required for the Min−Max calculation process. The calculation process is conducted using (7). Early initialization is conducted before entering the SMART and FCM calculation process stages on the proposed algorithm.
Initialization is needed to determine the results of student MBKM activity data clusters. The initial initialization data is shown in Table 2. For the SMART method, the weighting is done on the criteria data with a total value of 100%. Then for the Fuzzy C−Means method, the smallest expected error value will be 0.00001, and the power value is 2. Then three initial partition matrix data are generated randomly for each student data with a data value range of 0−1. And the sum of the three data values in each data will produce a value of 1. The first step is to normalize each criterion weight to obtain the relative weight by using (1). Then calculate the value of the evaluation factor for each alternative Jurnal Infotel, Vol. 15, No. 1, February 2023 https://doi.org/10.20895/infotel.v15i1.888 Assessment decisions of independent learning activities using SMART-FCM method with (2). From the value of the evaluation factor obtained, (3) is used to obtain the value of the evaluation weight for each alternative data [25], [26].
The result of the decision from the SMART method by adding up all the results of the assessment. From the total value, it will be a decision to group students based on predetermined values, including Cluster 1 (feasible) >= 0.7, Cluster 2 (considered) >= 0.5, and Cluster 3 (not feasible) < 0.5.
The student groups generated by the SMART method consist of 3 clusters where the first cluster has a total of 36 students, the second cluster has 14 students, and the third cluster has 2 students. The steps in the calculation using Fuzzy C−Means in the first iteration are the first time the initial three partition values are randomly set with a value range of 0−1 with a total of three values must be equal to 1, this is intended so that the results of data grouping are stable. The initial partition values are described in Table 5. The value of the evaluation weight that has been obtained from the SMART method in Table 3 is used in the calculation of the cluster center by calculating the degree of cluster membership. The cluster center is shown in Table 6. Calculate the change in the partition matrix in determining the membership of each student's data. Then calculate the objective function. The total value of the objective function is 0.19129.
The value of the objective function will determine whether the iteration will stop or continue by comparing it to the smallest expected error value. The value obtained is still greater than the smallest error value, which is 10 −5 , then the iteration continues, and the partition change value will be used as a partition calculation in the next iteration. The fuzzy C-Means calculation process stops at the 87th iteration where the objective function value is 0.00001. Student data grouping using FCM is shown in Table 7. The data group generated from the SMART−FCM method in the first cluster has data of 27 students, the second cluster has 16 students, and the third cluster has 10 students.

IV. DISCUSSION
In order to determine the best grouping results, the Silhouette Coefficient calculation is used for the results of the data group from the two methods. Based on the results of the Silhouette Coefficient calculation shown in Table 8, show that the SMART−FCM result has a value of 0.31187 which is better than using SMART alone because it has a value close to 1 even though the difference is not too far away. The results of the cluster data generated from the SMART and SMART−FCM methods have quite a significant difference, especially in the first and third clusters. However, the results obtained from grouping data using the SMART−FCM method are much more even, so the grouping data makes much more sense in making decisions based on the values obtained by students in independent campus activities.
This proves that the decision results using SMART will be more accurate using the FCM method. and the results of the decision can be used as a reference in assessing student MBKM activities.

V. CONCLUSION
From the research that has been conducted, it is found that the SMART−FCM method produces a better group of student data than the SMART results. Although the decision to group student data on SMART can be influenced by the specified value limit, FCM can group student data based on the characteristics and characteristics of members, making it much more efficient and precise. However, the results obtained are still unsatisfactory because the results of the silhouette coefficients have a value that is not too far away. Henceforth, research will be implementing the decision-making system method with other cluster algorithms to get better decision results.