The literature review presented in this research illustrated the importance of performance appraisal systems and their challenges. It also described the motivations of the goal setting process and the SMART approach for setting good and effective objectives. This thesis has shown how the developed system can support employee performance appraisals by facilitating the setting of SMART objectives and providing feedback based on the use of ILP and data mining techniques.
In general, performance appraisal has gained interest over the past few years and there are few studies that show interest in solving some problems of performance appraisal by applying automatic methods such as data mining methods for making significant decisions and classifying future data instances. For example, the study presented by Jantan et al. (2010) applies classification techniques in data mining to build a classification model which can be used for predicting the performance of employees in order to identify whether an employee is recommended for promotion or not based on his/her performance. Furthermore, the recent study proposed by Al-Radaideh and Al Nagi (2012) has also used data mining techniques to predict the performance of employees. This study utilises decision trees to build a classification model by using three datasets: personal information dataset (attributes: age, gender, marital status and number of kids), education information dataset (attributes: university type, general
specialisation, degree and grade) and professional information dataset (attributes: number of experience years, job title, rank and salary). The constructed classification model classifies the most important factors (attributes) that might have the highest effect on the employee performance such as job title.
As well as to these studies, there are some commercial systems for performance appraisals and goal setting that have been developed such as Halogen eAppraisal19,
GoalsOnTrack20 and Lifetick21. The Halogen eAppraisal software is a performance
appraisal system that allows managing the progress of employee performance appraisal process by reviewing the employee appraisal reports as well as tracking the progress of employees’ goals using goals report. Goal setting software such as GoalsOnTrack enables the user to write down his/her goals (or objectives) and facilitates the process of checking whether the goals are on track by tracking the progress of achieving the goals. The Lifetick software is a web based system which allows the user to create a set of goals (or objectives), tasks and reminders as well as enables him/her to track the progress of goals and tasks.
Even though the GoalsOnTrack and Lifetick systems support different tasks for setting goals and objectives, they do not have the ability to automatically assess whether an objective is SMART and do not have the potential for checking whether the resources and time are available to achieve the objective. Moreover, they do not perform any processing or analysing for the objectives text to automatically check the presence of the target words that are commonly used in writing SMART objectives as well as the presence of measures and dates in the objective sentences. Both systems assume that the user has knowledge about the SMART criteria, since a user of these systems has to ensure that an objective is SMART by himself/herself. For instance, the Lifetick software asks the user if he/she considers his/her objective as “specific”, “measurable”, “achievable”, “relevant”, and “time-specific”, and prompts him/her to tick the appropriate box for each element of the SMART criteria. In the GoalsOnTrack system, the user must follow the goal creation form instructions to create a SMART objective. However, this system is not able to assess whether a
19http://www.halogensoftware.com/products/halogen-eappraisal/ 20 http://www.goalsontrack.com
written objective is SMART, since the assessment process of the objective depends entirely on the user himself/herself.
Accordingly, the current commercial systems for setting objectives mainly focus on tracking objectives and do not support deeper aspects of the goal setting process such as assessing whether the objectives are SMART and providing feedback on the objectives.
In contrast, the developed system for this research is based on a novel framework that aims to support the writing of SMART objectives and providing feedback by performing the following tasks and contributions:
Analyse and process the written text of an objective to automatically check whether the essential elements for formulating SMART objectives are specified in the objective (e.g. target words, measures and dates).
Clarify the words that should be used when writing SMART objectives.
Predict the objective outcomes (e.g. amount of sales or costs) and estimate the required resources that must be allocated to achieve the given objective based on past historical data.
Automatically check whether an objective could be accomplished within the given resources and time based on past experience.
Automatically assess whether the written objectives are SMART.
Provide guidance and constructive feedback on structuring the objectives. Thus, the work done in this research is novel and complements previous research and existing commercial systems.
Table 21 presents a comparison between the developed system for this research and the GoalsOnTrack and Lifetick systems for objectives setting by showing the main characteristics of each system:
System Characteristics GoalsOnTrack Lifetick The Developed System
Enable a user to create objectives √ √ √
Automatically assess whether the written objectives are SMART
Χ Χ √
Process the written objectives text to automatically check whether an objective includes the required features in order to be “specific”, “measurable” and “time-related”
Χ Χ √
Automatically check whether an objective could be achieved within the given resources and time
Χ Χ √
Clarify the words that should be
used when writing SMART
objectives
Χ Χ √
Predict the objective outcomes and the resources that must be allocated to achieve the objective
Χ Χ √
Provide constructive feedback regarding the written objectives
Χ Χ √
Track the progress of the objectives √ √ Χ
Table 21 The Main Characteristics of the Developed System and Related Systems
6.5 Summary
In this chapter, a description of the system implementation was given. An explanation of some illustrative examples of objective sentences which have been assessed by the developed system for this research was presented in order to demonstrate how this system can be used to automate the process of setting SMART objectives and providing feedback based on AI techniques. The chapter included two empirical evaluations. First, an empirical evaluation for objectives from the sales domain was carried out. This included developing a corpus of objectives from the sales domain, and then applying the system on this corpus. The overall accuracy obtained for this corpus was 83%, with 33% TP rate and 50% TN rate. To test the generality of the framework, a second empirical evaluation for objectives from the costs domain was preformed. This included constructing another corpus of objectives from the costs domain, and then applying the system on this corpus. The overall accuracy achieved for the second corpus was 77%, with 23.3% TP rate and 53.3% TN rate. A comparison of the rules generated for both evaluations suggests that the approach is generic in terms of the checking whether the objectives are “specific”, “measurable” and “time-related”. The ability to assess if the objectives are “achievable” and “realistic” depends on the domain and prediction/classification methods used as well
as the availability of data. Where this is not possible, this part of the framework will not, of course, be applicable. A comparison between the developed system and related systems was given to show the novelty and the main contributions of the developed system for this research by highlighting its efficiency and effectiveness in setting SMART objectives and providing feedback.
In the next chapter, a conclusion that summarises the achievements made in this research and how the research objectives have been addressed will be presented including the results obtained through the experimental evaluation of ILP and data mining methods as well as the results of the two empirical evaluations of the developed system. Furthermore, a summary of the limitations of the framework and directions for future work will also be proposed.