MARCO CONCEPTUAL

3.3.4.1 Datasets and Experimental Setup

Experiments of the proposed item-oriented, user-oriented and density-oriented ap- proach (ESVD) are conducted on the classic recommender system datasets: the

Movielens 100K and the subset of theNetflix (the first 106,150 ratings are extract-

ed from the full Netflix dataset as the subset of Netflix, which are made by 1,910 users on 1,780 movies). Some experiments with the larger version are also performed and obtained similar results. However, it requires much longer time to perform the experiments since the models are trained and tested each time for different choice ofNandN0. Therefore, the smaller datasetsMovielens 100K and subset of original

Netflix are focused to be able to run more experiments, in order to explore how these

two parameters affects the results of the proposed matrix factorization methods. Normally each dataset is partitioned into a training set and a test set. The model is trained on the training set and the quality of results is usually measured by the Root Mean Square Error (RMSE) of the test set. RMSE is used as the default metric, which is widely used in theNetfilx Competition [1] and proved to be effective for measuring recommender systems.

The number of the latent factors (rank) kare set to be 10 for training each matrix factorization model. Although increasing it does raise the performance, the computational cost is proportional to latent factors. For matrix factorization of the sub-matrix, the coefficient of the regularization term ku and kv are 0.01 and 0.05

0 5% 10% 15% 20% 25% 30%

the percentage of items and users selected in the block matrix

0.96 0.97 0.98 0.99 1 RMSE Item-oriented User-oriented Density-oriented

Figure 3.6: Movielens: RMSE comparisons of proposed methods based onSVD

0 5% 10% 15% 20% 25% 30%

the percentage of items and users selected in the block matrix

0.925 0.93 0.935 0.94 RMSE Item-oriented User-oriented Density-oriented

Figure 3.7: Netflix: RMSE comparisons of proposed methods based onSVD

α is 0.1 with a decrease by a factor of 0.9 each iteration for both datasets. For matrix factorization of the rating matrix R0 (with pre-estimations), the coefficient of regularization term ku0 and kv0 are 0.1 for both datasets, and the learning rate

α is 0.01 and 0.05 with decrease by a factor of 0.9 each iteration for the Movielens

Table 3.1: RMSE ofESVD onMovielens 100K (The Density-Oriented Approach)

Items&Users Block Density Extra Ratings RMSE

0% null null 0.9709

5% 77.28% 897 0.9677

10% 65.20% 5496 0.9632

15% 53.90% 16381 0.9630

20% 45.66% 34508 0.9570

Table 3.2: RMSE ofESVD onNetflix (The Density-Oriented Approach)

Items&Users Block Density Extra Ratings RMSE

0% null null 0.9306 5% 59.06% 3498 0.9265 10% 43.59% 19179 0.9265 15% 33.10% 51268 0.9291 20% 25.51% 101298 0.9319 3.3.4.2 Experimental Results

Figure 3.6 and Figure 3.7 show the results of the proposed methods based on how many items and users selected (simply setting N = N0 in this case) in the sub- matrix on theMovielens 100K and Netflix datasets, respectively. All the methods start at 0 point where no extra filling is added into the learning process, which is the same as RSVD. It can be seen that the results of item-oriented approach and user-oriented sometimes are not promising. Because in the item-oriented (or user-oriented) approach only pre-estimations are added based on the most popular movies (or users), which may lead to a lot of bias and distort the latent factor model. For example, most people prefer happy endings, and the consequence is that comedies are more popular than tragedies. As a result, a lot of comedy movies would be elicited for each user to give ratings which leads to more weights on the factor corresponding to comedies in the latent factor model (RSVD in this case). It is apparent from the Figure 3.6 and Figure 3.7 that the proposedESVD consistently outperforms other methods including the baseline method: RSVD.

oriented (ESVD) method are illustrated which incorporates both item-oriented and user-oriented approach on the Movielens 100K and Netflix datasets. Different

RMSE are compared based on how many items and users (N = N0 from 0% to 20%) selected. Note that the basic matrix factorization is a special case of the pro- posed method when settingN= 0%, which is used as the baseline for comparision. After selecting a certain percentage of items and users, a sub-matrix is formed. It can be observed that the more items and users are chosen, the much sparser the sub-matrix is. The missing values in the sub-matrix are chosen to be pre-estimated ratings. Although sparser matrix may lead to a less accurate matrix factorization model and the quality of pre-estimations may not as good as the ones from the denser matrix, the number is increased. Therefore more ratings can be obtained and put into the process of learning the target matrix factorization model. At last predictions are computed on the test set and corresponding results are obtained. Because the sub-matrix is the intersection of the largest N items and N0 users, its density is much greater than the one from item-oriented or user-oriented approach. Even with fewer ratings to be added compared with item-oriented and user-oriented, the results are better.

In the experiments, it can be observed that for theMovielens 100K dataset the performance fluctuates as the number of projects increases (Figure 3.6). While for the Netflix dataset (Figure 3.7), the performance drops at first then it dete- riorates (the lower RMSE the better performance) as N goes up. This is mainly because theNetflix dataset is much sparser than theMovielens 100K dataset. While adopting theESVD algorithm, asN increases, more poor quality data is added into the learning process and leads to the distortion of the model (Figure 3.7). The op- timal point (N) that balances the quality (density of sub-matrix) and the quantity (number of added ratings) depends on the distribution of ratings. For theMovielens 100K dataset, the proposedESVDcan reach 0.9570 (whenN= 20%) which reduces theRMSE by 0.0139 compared with the Regularized SVD 0.9709. For the Netflix

dataset, it could lower theRMSE by 0.0047 (from 0.9306 to 0.9259 when N= 3%).