Towards More Confident Recommendations: Improving Recommender Systems Using Filtering Approach Based on Rating Variance
|
|
- Samantha Stafford
- 5 years ago
- Views:
Transcription
1 Towards More Confident Recommendations: Improving Recommender Systems Using Filtering Approach Based on Rating Variance Gediminas Adomavicius Sreeharsha Kamireddy 2 skamir@cs.umn.edu YoungOk Kwon ykwon@csom.umn.edu Information and Decision Sciences, Carlson School of Management, University of Minnesota 2 Department of Computer Science and Engineering, University of Minnesota Abstract. In the present age of information overload, it is becoming increasingly harder to find relevant content. Recommender systems have been introduced to help people deal with these vast amounts of information and have been widely used in research as well as e-commerce applications. In this paper, we propose several new approaches to improve the accuracy of recommender systems by using rating variance to gauge the confidence of recommendations. We then empirically demonstrate how these approaches work with various recommendation techniques. We also show how these approaches can generate more personalized recommendations, as measured by the coverage metric. As a result, users can be given a better control to choose whether to receive recommendations with higher coverage or higher accuracy.. Introduction and Motivation In the present age of information overload, it is becoming increasingly harder to find relevant content. This problem is not only widespread but also alarming [9]. Over the last decade, recommender systems have been introduced to help people deal with these vast amounts of information [2]. Recommender systems have been widely used in research as well as e-commerce applications, such as MovieLens, Amazon, Travelocity, and Netflix. Most common formulation of the recommendation problem relies on the notion of ratings, i.e., recommender systems estimate ratings of items that are yet to be consumed by the users (based on the ratings of items already consumed). Recommendations to users are typically made based on the strength of the predicted ratings, i.e., the items with the highest predicted ratings are the ones being recommended. However, it has been argued that, in addition to recommendation strength, presenting the confidence of a recommendation (i.e., how certain the system is about the accuracy of the recommendation) to a user may be no less important [7]. Nonetheless, the issue of confidence has not been explored comprehensively, and the vast majority of recommender systems provide recommendations based solely on their strength. However, high-strength predictions may not always mean good accuracy, since, for example, they are often obtained from small amounts of data. Therefore, beyond the recommendation of simply high predicted ratings, we need to provide recommendations that users can rely on with confidence. A confidence measure is important as it can help users decide which movies to watch, products to buy, and also help an e-commerce site in making a decision on which recommendations should not be displayed, because an erratic recommendation can diminish the trust of users in the system [7]. Surveys of MovieLens users also indicate the need for a proper confidence display [6], and it has been demonstrated how the display of confidence measures has helped the users in making better decisions and the need for more sophisticated confidence measures has also been expressed []. A wide range of recommendation algorithms have been developed, but hardly any of the methods give a measure of confidence []. In this paper, we propose several new approaches to improve the accuracy of recommendations by using rating variance (which, as we show, is inversely related to the recommendation accuracy) to gauge the confidence of recommendations. We then empirically show how these approaches work with different recommendation techniques. We also show how these approaches can generate more personalized recommendations, as measured by the coverage metric (described later in the paper in more detail). As a result, users can be given a better control to choose whether to receive recommendations with higher coverage or accuracy. 2. Related Work Recommender systems are usually classified into three categories based on their approach to
2 recommendation: content-based, collaborative, and hybrid approaches [2]. Content-based recommender systems recommend items similar to the ones the user preferred in the past. Collaborative (or collaborative filtering) recommender systems recommend items that users with similar preferences have liked in the past. Finally, hybrid approaches can combine content-based and collaborative methods in several different ways []. Furthermore, recommender systems can also be classified based on the nature of their algorithmic technique into memory- and model-based approaches [4]. Memory-based techniques usually represent heuristics that calculate recommendations on the fly based directly on the previous user activities. In contrast, model-based techniques use previous user activities to first learn a predictive model (typically using some statistical or machine-learning methods), which is then used to make recommendations. Numerous recommendation techniques have been developed over the last few years, and several different metrics have been employed for measuring recommendation accuracy, hardly any of the techniques provide the measure of confidence []. To address this, [] have used a number of ratings submitted for each item as a simple, non-personalized measure of confidence for predictions in a recommender system. We use this as a starting point to develop richer confidence metrics and suggest ways of giving control on confidence to the user. It has also been shown that explaining recommendations was effective in helping users adopt recommendations [6, ]. In particular, in our proposed approaches we use rating variance for each user and/or item, since each user/item may exhibit different rating habit/pattern which can affect the accuracy of prediction. This motivation on using variance is shared in the decision science literature []. When a decision maker is provided with multiple opinions and needs to aggregate them, the variance of the individuals opinions is used as a convenient proxy of decision maker s confidence in their opinion. A somewhat similar approach in recommender systems literature is used in [8], where the variance in rating patterns among the users with similar interests is captured using a decoupled model. That is, they develop a specialized model-based technique to account for rating patterns. In contrast, our approaches can be used in conjunction with any existing recommendation technique. We can simply employ our new rating-variance-based filtering methods to the predictions already obtained from any existing technique, resulting in the improved recommendation accuracy. However, while better understanding of the recommendation confidence can improve the recommendation accuracy, the accuracy alone may not be enough in evaluating the performance of recommender systems; e.g., it is easy to obtain higher precision by recommending only very popular items that most users are likely to rate highly. If we recommend items based only on the popularity of items, then there is no personalization, and the recommender systems will be of no use. It has often been suggested that recommender systems must be not only accurate, but also useful [7]. Thus, various coverage-based usefulness measures (e.g., measuring the percentage of items that the recommender system is able to make recommendations for) are used to make sure that the recommendations are personalized with enough item variety [7]. For example, as a variant of coverage measure, [] proposed a new metric which computes an average dissimilarity between all pairs of recommended items. Our proposed recommendation approaches will also be evaluated with a coverage measure (in addition to accuracy) and will demonstrate how to employ the notion of confidence to assure certain levels of both accuracy and coverage.. New Rating-Variance-Based Filtering Approaches We propose several techniques that can exploit the confidence of recommendations while using any existing recommendation algorithm as a black box. We first predict the ratings of unrated items with some existing collaborative filtering techniques. We then analyzed how the accuracy of predictions is related to various user/item statistics, such as rating frequency, mean, variance, and predicted rating value for different users and items. Among the statistics, we consistently found that the recommendation accuracy monotonically decreases (i.e., the mean absolute error increases) as the rating variance of the user/item increases. As a result, we have learned that, if we recommend to users only those items whose rating variance is small, the accuracy of recommendations can improve.
3 Recommender systems typically recommend the most relevant N items to each user, since the users in real-world personalization applications are usually interested in looking at only several highestranked item recommendations. We used movie recommendation dataset in our experiments, and the movie ratings in this data were on a scale to ; therefore, we first needed to define what highly-ranked means. We split the ratings into a binary scale by treating ratings greater than. (i.e., ratings 4 and ) as highly-ranked and ratings less than. (i.e., ratings, 2, and ) as non-highly-ranked. Thus, the rating values of all recommended items should be greater than the threshold of.. We then evaluated the accuracy of the recommender systems based on the percentage of truly high ratings among those that were predicted to be the N most relevant items for each user (i.e., using the precision-in-top-n metric).. Simple Filtering Approach In general, top-n items for each user are obtained in two steps: filtering the predicted ratings greater than the pre-defined acceptable threshold (e.g.,. out of ), and then choosing N items above the threshold with the highest predicted ratings. In our first simple filtering approach, in addition to filtering based on the acceptable rating threshold, we also filter out the recommendations according to some user-specified ratings standard deviation threshold D. In particular, for each rating (of an item to a user) that is predicted by the recommender system, we also assign a variance score which, in our case, is calculated as the standard deviation of all known ratings for that predicted item. In our experiments, we varied threshold D from to.2 in increments of. (note, that the average rating std. dev. of items was approximately. in our dataset). E.g., in the case of D =, we recommended to users only items with a rating std. dev. of less than (and, of course, where the predicted rating was above.). We also tested the standard recommendation approach, i.e., without any constraints on the rating std. dev. (D = ), in order to compare it with the proposed approaches. With this simple filtering approach, only the highly-ranked items with low rating std. dev. will be recommended to users, and in turn, we expect that the more the rating std. dev. threshold restricted the recommended items, the better the performance of the recommendations would be in terms of the precision-in-top-n metric. However, as mentioned above, recommender systems must be not only accurate but also useful [7]. As a measure of usefulness, we use a coverage metric by computing the total number of unique items recommended across all users. This coverage metric is as useful a measurement as precision, because it reflects the level of personalization realized by the system; i.e., this coverage metric will show whether all users are recommended the same top-n items or every user is given his/her own unique top-n items (or something in-between). One can expect there to be a tradeoff between accuracy and coverage it may be easy to achieve fairly good accuracy (in terms of precision-in-top-n) by providing recommendations of the same most popular items to everybody, but the coverage (i.e., the level of personalization) will clearly suffer, and vice versa. Therefore, it is important to maintain a balance between accuracy and coverage. To address this issue, we propose the smart and safe filtering approaches that can increase the coverage of recommendations while maintaining a relatively high precision-in-top-n..2 Smart and Safe Approaches Smart and safe approaches directly use a rating std. dev. (of each item) in selecting recommendations by adjusting the predicted rating values instead of using a filtering threshold. In particular, in the smart approach, when predicting the rating for item x for a given user, we take the predicted rating generated by any traditional recommender system and subtract one standard deviation of known ratings of item x from this predicted rating. In other words, this way we try to model a worst case scenario i.e., how low the actual rating could be, if the prediction by the traditional recommender system is not very accurate. We then recommend the item, if this newly computed predicted rating is greater than the acceptable rating threshold (i.e.,.). Specifically, we chose the distance of one standard deviation from the predicted rating as a worst case because it usually accounts for a significant portion of data (e.g., if the data is normally distributed, one standard deviation from the mean accounts for 68% of the sample). That is, we reduce the rating by size of one standard deviation and check to see whether this adjusted value can still be considered as highly-ranked. If so, we recommend N items, according to the order of the original (not adjusted) predicted ratings. I.e., we rank the top-n items based on the average case scenario, but
4 make sure that we recommend only those items that would be above the acceptable threshold in the worst case scenario. The safe approach is a slight variation of smart, only slightly more conservative. Here, after subtracting one standard deviation from the predicted ratings, we rank top-n items based on the newly adjusted ratings (i.e., worst case scenario) instead of the original predicted ones (i.e., average case scenario). Both safe and smart approaches assure that ratings adjusted by one standard deviation still exceed the acceptable rating threshold (i.e.,.). Thus, recommendations should arguably be fairly accurate (in terms of precision-in-top-n), while their coverage does not deteriorate as much as in the simple filtering approach (as we empirically demonstrate below). All three new approaches are summarized in Figure.. PRE-PROCESSING: Predict the ratings for each user Known Ratings R Any Recommendation Technique Predicted Ratings R' 2. FILTERING: Filter R' based on rating std. dev. Simple Filtering Approach R' >= ART Std. dev. of R < D (D =,,,.2,.) Smart Approach R' ( std. dev.) >= ART Safe Approach R' ( std. dev.) >= ART R' := R' ( std. dev.). POST-PROCESSING: Recommend top N items for each user Sort R' in a descending order Choose top N items with the highest rating R' for each user Figure. Three rating-variance-based filtering approaches. (ART = Acceptable Rating Threshold =.) 4. Empirical Results To evaluate the proposed approaches, we used a MovieLens ratings dataset (available at with, movie ratings by various users. We prepossessed the data in order to obtain enough ratings for the predictions by including only the ratings by users who rated at least 2 movies and the ratings of movies rated by at least 2 users. We ended up with 94,44 ratings, including 97 users and 97 movies (i.e., data sparsity %). As mentioned earlier, we can use any of the existing recommendation techniques in conjunction with our proposed approaches. As an example of a memory-based technique, we chose one of the most popular collaborative filtering (CF) techniques that uses an adjusted weighted sum and a cosine similarity metric [4]. A Naïve Bayesian classifier was chosen as the model-based technique; despite its simplicity, it is competitive with other learning algorithms in many cases []. For Naïve Bayesian classifier, we extended the model mentioned in [4] to multi-class rating data (i.e., scale -) in our experiment. Both user-based and item-based approaches with varying neighborhood sizes were examined for both the memory- and model-based techniques, and we obtained similar results in all cases. Here, because of space limitations, we display only the results of the item-based CF technique with the neighborhood of all users. Figure 2 shows the results of the three proposed approaches in both precision (line graph) and coverage (bar chart) for top-n items for each user (where N =,, ) using the memory- and model-based techniques. The simple filtering approach (with rating std. dev. threshold D =,,,.2, and ) clearly shows that a lower std. dev. threshold leads to higher precision, but lower coverage. In addition, the smart and safe approaches compare favorably to the simple filtering approach in terms of performance. In particular, in conjunction with the memory-based technique, the smart and safe approaches demonstrate the highest precision and have relatively high coverage. Furthermore, in conjunction with
5 the model-based technique, these two approaches demonstrate very high coverage and have relatively high precision. Note that, in general, the model-based technique produces higher coverage than the memory-based one across all proposed variance-based filtering approaches. This can be explained by the fact that the predicted rating values acquired from the model-based technique are discrete, not continuous. That is, many predictions will have identical values (e.g., rated as ), and the top-n items are chosen randomly from this list of highest-rated items for each user, thus resulting in higher variety. Also, as expected, because of its more conservative sorting procedure based on the worst-case scenario, the safe approach performs better in precision and is similar or worse in coverage than the smart approach. From the results, we can conclude that the smart and safe approaches have a more balanced performance in their precision and coverage metrics, as compared to the simple filtering approach. Memory-based (adjusted weighted sum and cosine similarity metric) technique Safe Smart....2 Any std dev Model-based (Naïve Bayesian Classifier) technique Safe Smart....2 Any std dev Figure 2. Precision and coverage in top-n (N =,, ) of the three proposed approaches (precision-in-top-n: line graph with the right Y-axis; coverage: bar chart with the left Y-axis ) We also tested various portions of one standard deviation in the range of to in increments of. (i.e., to 68% confidence intervals) in the smart and safe approaches, since one standard deviation was only one way to consider rating variance. We replaced the R' std. dev. formula in Figure with R' k * std. dev (k =,.,, ). If k is, then there is no rating variance constraint. Figure shows that precision increases and coverage decreases as we subtract a larger portion of one standard deviation, and that the change rate of precision and coverage are lower in the model-based technique than in the memory-based technique. In summary, this result is consistent with the result of Figure 2, i.e., the stricter the rating variance constraint is, the higher precision and the lower coverage is obtained. This is especially useful in practical recommendation applications, since users can have a better control to choose whether to receive recommendations with higher coverage (i.e., more personalized) or higher accuracy by choosing portions of the standard deviation as filtering thresholds. Smart approach with Memory-based technique.7.7 Safe approach with Model-based technique Figure. The smart and safe approaches with various standard deviations (,.,, ).. Conclusions, Discussion, and Future Work In this paper, using a simple filtering approach we have demonstrated that prediction accuracy can be
6 significantly improved by filtering out recommendations above a minimum rating std. dev. threshold. However, there was also a corresponding decrease in the coverage of recommendations. We then proposed the smart and safe approaches which generate recommendations of greater value by providing a good balance of prediction accuracy and coverage. New approaches are especially useful, since they can confidently improve the accuracy of recommendation, and in addition to that, a user can control the balance between the accuracy and coverage of recommendations. Furthermore, this research provides some interesting opportunities for future work. In particular, the rating variance can be considered in selecting the training data for unknown rating prediction, which is typically referred to as an active learning problem. If we can choose ratings with potentially high accuracy (i.e., ratings with a low average of user rating variance and item rating variance) among all the known ratings, we may arguably reach the desired accuracy level faster (i.e., with less training data) than when we choose the data randomly. Our preliminary results show that this active learning approach (which uses only a fraction of available ratings, i.e., the ones with lower variance) increases the performance in accuracy metrics such as mean absolute error, precision, and recall in the earlier stages, although a more comprehensive further exploration is still needed. Also, we are currently extending our proposed approaches to more advanced model-based techniques, such as Flexible Mixture Model [4], and plan on testing them on additional datasets, such as the ones from Yahoo Movies and Netflix, as well as with other evaluation metrics. Finally, the proposed approaches currently can be applied as a postprocessing step to any recommendation technique. However, these approaches could be more directly integrated into some existing recommendation techniques, including traditional neighborhood-based collaborative filtering [4, ], where the variance of neighborhood ratings (as opposed to all ratings for the same item) could be exploited to further improve the recommendation quality. Acknowledgments Work reported in this paper was supported in part by the National Science Foundation grant no References [] G. Adomavicius, A. Tuzhilin, Toward the Next Generation of Recommender Systems: A Survey of the Stateof-the-Art and Possible Extensions, IEEE Trans. on Knowledge and Data Engineering, 7(6):74-749,. [2] M. Balabanovic, Y. Shoham, Fab: Content-Based, Collaborative Recommendation, Comm. ACM, 4():66-72, 997. [] K. Bradley and B. Smyth, Improving Recommendation Diversity, Proc. of 2 th Irish Conf. on Artificial Intelligence and Cognitive Science,. [4] J.S. Breese, D. Heckerman, C. Kadie, Empirical Analysis of Predictive Algorithms for Collaborative Filtering, Proceedings of 4 th Conference on Uncertainty in Artificial Intelligence, 998. [] D.V. Budescu, A.K. Rantilla, H.T. Yu, and T.M. Karelitz, The effects of asym-metry among advisors on the aggregation of their opinions, Organizationl Behavior and Human Decision Processes, 9:78-94,. [6] J.L. Herlocker, J.A. Konstan, J. Riedl, Explaining collaborative filtering recommendations, Proc. of the Conf. Computer Supported Cooperative Work,. [7] J.L. Herlocker, J.A. Konstan, L.G. Terveen, J. Riedl, Evaluating Collaborative Filtering Recommender Systems, ACM Transactions on Information Systems, 22():-, 4. [8] R. Jin, L. Si, C.X. Zhai, J.Callan, Collaborative Filtering with Decoupled Models for Preferences and Ratings, Proc. of 2 th Intl. Conf. on Information and Knowledge management, [9] W. Knight, Info-mania' dents IQ more than marijuana, NewScientist.com news,. URL: [] S.M. McNee, S.K. Lam, C. Guetzlaff, J.A. Konstan, J. Riedl, Confidence Displays and Training in Recommender Systems, Proc. of IFIP TC Intl. Conf. on Human-Computer Interaction,. [] T. Mitchell, Machine Learning, McGraw-Hill, 997. [2] P. Resnick, H. R. Varian, Recommender systems, Comm. ACM, 4():6 8, 997. [] B. Sarwar, G. Karypis, J. Konstan, J. Riedl, Item-Based Collaborative Filtering Recommendation Algorithms, Proc. of th International WWW Conference,. [4] L. Si, R. Jin, Flexible Mixture Model for Collaborative Filtering. Proc. of 2 th Intl. Conf. on Machine Learning,.
Overcoming Accuracy-Diversity Tradeoff in Recommender Systems: A Variance-Based Approach
Overcoming Accuracy-Diversity Tradeoff in Recommender Systems: A Variance-Based Approach Gediminas Adomavicius gedas@umn.edu YoungOk Kwon ykwon@csom.umn.edu Department of Information and Decision Sciences
More informationOn the Combination of Collaborative and Item-based Filtering
On the Combination of Collaborative and Item-based Filtering Manolis Vozalis 1 and Konstantinos G. Margaritis 1 University of Macedonia, Dept. of Applied Informatics Parallel Distributed Processing Laboratory
More informationA Comparison of Collaborative Filtering Methods for Medication Reconciliation
A Comparison of Collaborative Filtering Methods for Medication Reconciliation Huanian Zheng, Rema Padman, Daniel B. Neill The H. John Heinz III College, Carnegie Mellon University, Pittsburgh, PA, 15213,
More informationCollaborative Filtering with Multi-component Rating for Recommender Systems
Collaborative Filtering with Multi-component Rating for Recommender Systems Nachiketa Sahoo Ramayya Krishnan George Duncan James P. Callan Heinz School Heinz School Heinz School Language Technology Institute
More informationStability of Collaborative Filtering Recommendation Algorithms 1
Stability of Collaborative Filtering Recommendation Algorithms GEDIMINAS ADOMAVICIUS, University of Minnesota JINGJING ZHANG, University of Minnesota The paper explores stability as a new measure of recommender
More informationDe-Biasing User Preference Ratings in Recommender Systems
Gediminas Adomavicius University of Minnesota Minneapolis, MN gedas@umn.edu De-Biasing User Preference Ratings in Recommender Systems Jesse Bockstedt University of Arizona Tucson, AZ bockstedt@email.arizona.edu
More informationITEM-LEVEL TURST-BASED COLLABORATIVE FILTERING FOR RECOMMENDER SYSTEMS
ITEM-LEVEL TURST-BASED COLLABORATIVE FILTERING FOR RECOMMENDER SYSTEMS Te- Min Chag Department of Information Management, National Sun Yat-sen University temin@mail.nsysu.edu.tw Wen- Feng Hsiao Department
More informationThe Long Tail of Recommender Systems and How to Leverage It
The Long Tail of Recommender Systems and How to Leverage It Yoon-Joo Park Stern School of Business, New York University ypark@stern.nyu.edu Alexander Tuzhilin Stern School of Business, New York University
More informationA Social Curiosity Inspired Recommendation Model to Improve Precision, Coverage and Diversity
A Social Curiosity Inspired Recommendation Model to Improve Precision, Coverage and Diversity Qiong Wu, Siyuan Liu, Chunyan Miao, Yuan Liu and Cyril Leung Joint NTU-UBC Research Centre of Excellence in
More informationSawtooth Software. MaxDiff Analysis: Simple Counting, Individual-Level Logit, and HB RESEARCH PAPER SERIES. Bryan Orme, Sawtooth Software, Inc.
Sawtooth Software RESEARCH PAPER SERIES MaxDiff Analysis: Simple Counting, Individual-Level Logit, and HB Bryan Orme, Sawtooth Software, Inc. Copyright 009, Sawtooth Software, Inc. 530 W. Fir St. Sequim,
More informationInvestigating Confidence Displays for Top-N Recommendations
Investigating Confidence Displays for Top-N Recommendations Guy Shani & Lior Rokach & Bracha Shapira & Sarit Hadash & Moran Tangi Information Systems Engineering, Ben Gurion University Recommendation systems
More informationUsing Personality Information in Collaborative Filtering for New Users
Using Personality Information in Collaborative Filtering for New Users Rong Hu Human Computer Interaction Group Swiss Federal Institute of Technology (EPFL) CH-1015, Lausanne, Switzerland rong.hu@epfl.ch
More informationRecommender Systems in Computer Science and Information Systems - a Landscape of Research
Recommender Systems in Computer Science and Information Systems - a Landscape of Research Dietmar Jannach 1, Markus Zanker 2, Mouzhi Ge 3, and Marian Gröning 1 1 TU Dortmund, 44227 Dortmund, Germany {Dietmar.Jannach,
More informationResults & Statistics: Description and Correlation. I. Scales of Measurement A Review
Results & Statistics: Description and Correlation The description and presentation of results involves a number of topics. These include scales of measurement, descriptive statistics used to summarize
More informationA NOVEL VARIABLE SELECTION METHOD BASED ON FREQUENT PATTERN TREE FOR REAL-TIME TRAFFIC ACCIDENT RISK PREDICTION
OPT-i An International Conference on Engineering and Applied Sciences Optimization M. Papadrakakis, M.G. Karlaftis, N.D. Lagaros (eds.) Kos Island, Greece, 4-6 June 2014 A NOVEL VARIABLE SELECTION METHOD
More informationHow to Create Better Performing Bayesian Networks: A Heuristic Approach for Variable Selection
How to Create Better Performing Bayesian Networks: A Heuristic Approach for Variable Selection Esma Nur Cinicioglu * and Gülseren Büyükuğur Istanbul University, School of Business, Quantitative Methods
More informationThe Predictive Power of Bias + Likeness Combining the True Mean Bias and the Bipartite User Similarity Experiment to Enhance Predictions of a Rating
The Predictive Power of Bias + Likeness Combining the True Mean Bias and the Bipartite User Similarity Experiment to Enhance Predictions of a Rating Hongze Lai (laihz) Neel Murthy (nmurthy) Nisha Garimalla
More informationHybridising collaborative filtering and trust-aware recommender systems
Hybridising collaborative filtering and trust-aware recommender systems Charif Haydar, Anne Boyer, Azim Roussanaly To cite this version: Charif Haydar, Anne Boyer, Azim Roussanaly. Hybridising collaborative
More informationRecommender Systems, Consumer Preferences, and Anchoring Effects
Recommender Systems, Consumer Preferences, and Anchoring Effects Gediminas Adomavicius University of Minnesota Minneapolis, MN gedas@umn.edu Jesse Bockstedt George Mason University Fairfax, VA jbockste@gmu.edu
More informationThe 29th Fuzzy System Symposium (Osaka, September 9-, 3) Color Feature Maps (BY, RG) Color Saliency Map Input Image (I) Linear Filtering and Gaussian
The 29th Fuzzy System Symposium (Osaka, September 9-, 3) A Fuzzy Inference Method Based on Saliency Map for Prediction Mao Wang, Yoichiro Maeda 2, Yasutake Takahashi Graduate School of Engineering, University
More informationTypicality-based Collaborative Filtering Recommendation
1 Typicality-based Collaborative Filtering Recommendation Yi Cai, Ho-fung Leung, Qing Li, Huaqing Min, Jie Tang and Juanzi Li School of Software Engineering, South China University of Technology, Guangzhou,
More informationE-MRS: Emotion-based Movie Recommender System
E-MRS: Emotion-based Movie Recommender System Ai Thanh Ho Ilusca L. L. Menezes Yousra Tagmouti Department of Informatics and Operations Research University of Montreal, Quebec, Canada E-mail: {hothitha,lopedei,tagmouty}@iro.umontreal.ca
More informationAlleviating the Sparsity in Collaborative Filtering using Crowdsourcing
Alleviating the Sparsity in Collaborative Filtering using Crowdsourcing Jongwuk Lee, Myungha Jang, Dongwon Lee The Pennsylvania State University, PA, USA {jxl90,dongwon}@psu.edu Won-Seok Hwang, Jiwon Hong,
More informationThe Effectiveness of Personalized Movie Explanations: An Experiment Using Commercial Meta-data
The Effectiveness of Personalized Movie Explanations: An Experiment Using Commercial Meta-data Nava Tintarev and Judith Masthoff University of Aberdeen, Aberdeen, U.K. Abstract. This paper studies the
More informationClustering mass spectrometry data using order statistics
Proteomics 2003, 3, 1687 1691 DOI 10.1002/pmic.200300517 1687 Douglas J. Slotta 1 Lenwood S. Heath 1 Naren Ramakrishnan 1 Rich Helm 2 Malcolm Potts 3 1 Department of Computer Science 2 Department of Wood
More informationSawtooth Software. The Number of Levels Effect in Conjoint: Where Does It Come From and Can It Be Eliminated? RESEARCH PAPER SERIES
Sawtooth Software RESEARCH PAPER SERIES The Number of Levels Effect in Conjoint: Where Does It Come From and Can It Be Eliminated? Dick Wittink, Yale University Joel Huber, Duke University Peter Zandan,
More informationGePuTTIS: General Purpose Transitive Trust Inference System for Social Networks
GePuTTIS: General Purpose Transitive Trust Inference System for Social Networks Syavash Nobarany 1, Mona Haraty 1, Dan Cosley 2 1 School of ECE, University of Tehran, Tehran 14395-515, Iran 2 HCI Lab,
More informationDetermining the Vulnerabilities of the Power Transmission System
Determining the Vulnerabilities of the Power Transmission System B. A. Carreras D. E. Newman I. Dobson Depart. Fisica Universidad Carlos III Madrid, Spain Physics Dept. University of Alaska Fairbanks AK
More informationCOMP329 Robotics and Autonomous Systems Lecture 15: Agents and Intentions. Dr Terry R. Payne Department of Computer Science
COMP329 Robotics and Autonomous Systems Lecture 15: Agents and Intentions Dr Terry R. Payne Department of Computer Science General control architecture Localisation Environment Model Local Map Position
More informationCorrecting Popularity Bias by Enhancing Recommendation Neutrality
Correcting Popularity Bias by Enhancing Recommendation Neutrality Toshihiro Kamishima *, Shotaro Akaho *, Hideki Asoh *, and Jun Sakuma ** *National Institute of Advanced Industrial Science and Technology
More informationCase Studies of Signed Networks
Case Studies of Signed Networks Christopher Wang December 10, 2014 Abstract Many studies on signed social networks focus on predicting the different relationships between users. However this prediction
More informationGroup recommender systems: exploring underlying information of the user space
2013 BRICS Congress on 1st Computational BRICS Countries Intelligence Congress & on 11th Computational Brazilian Congress Intelligence on Computational Intelligence Group recommender systems: exploring
More informationCOLLABORATIVE filtering (CF) is an important and popular
IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 26, NO. X, XXXXXXX 2014 1 Typicality-Based Collaborative Filtering Recommendation Yi Cai, Ho-fung Leung, Qing Li, Senior Member, IEEE, Huaqing
More informationExploiting Implicit Item Relationships for Recommender Systems
Exploiting Implicit Item Relationships for Recommender Systems Zhu Sun, Guibing Guo, and Jie Zhang School of Computer Engineering, Nanyang Technological University, Singapore School of Information Systems,
More informationIEEE Proof Web Version
IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY, VOL. 4, NO. 4, DECEMBER 2009 1 Securing Rating Aggregation Systems Using Statistical Detectors and Trust Yafei Yang, Member, IEEE, Yan (Lindsay)
More informationRemarks on Bayesian Control Charts
Remarks on Bayesian Control Charts Amir Ahmadi-Javid * and Mohsen Ebadi Department of Industrial Engineering, Amirkabir University of Technology, Tehran, Iran * Corresponding author; email address: ahmadi_javid@aut.ac.ir
More informationIdentity Verification Using Iris Images: Performance of Human Examiners
Identity Verification Using Iris Images: Performance of Human Examiners Kevin McGinn, Samuel Tarin and Kevin W. Bowyer Department of Computer Science and Engineering University of Notre Dame kmcginn3,
More informationFramework for Comparative Research on Relational Information Displays
Framework for Comparative Research on Relational Information Displays Sung Park and Richard Catrambone 2 School of Psychology & Graphics, Visualization, and Usability Center (GVU) Georgia Institute of
More informationAn Efficient Service Rating Forecasting by Exploring Social Mobile User s Geographical Locations
An Efficient Service Rating Forecasting by Exploring Social Mobile User s Geographical Locations P. Sarika 1, Dr. A.P. Siva Kumar 2 P.G. Student, Department of CSE, JNTUA College of Engineering, Ananthapuramu,
More informationYou re special, but it doesn t matter if you re a greenhorn: Social recommender strategies for mere mortals
You re special, but it doesn t matter if you re a greenhorn: Social recommender strategies for mere mortals Pantelis P. Analytis (analytis@mpib-berlin.mpg.de) Daniel Barkoczi (barkoczi@mpib-berlin.mpg.de)
More informationINTRODUCTION TO STATISTICS SORANA D. BOLBOACĂ
INTRODUCTION TO STATISTICS SORANA D. BOLBOACĂ OBJECTIVES Definitions Stages of Scientific Knowledge Quantification and Accuracy Types of Medical Data Population and sample Sampling methods DEFINITIONS
More informationLARS*: An Efficient and Scalable Location-Aware Recommender System
TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 6, NO. 1, NOVEMBER 2012 1 : An Efficient and Scalable Location-Aware Recommender System Mohamed Sarwt, Justin J. Levandoski, Ahmed Eldawy and Mohamed
More informationUnderstanding user preferences and goals in recommender systems
Understanding user preferences and goals in recommender systems Martijn Willemsen Human-Technology Interaction EnCHIReS Workshop ACM EICS 26 June 2017 Netflix tradeoffs popularity, diversity and accuracy
More informationCannabis Culture Poll. November 2018
Cannabis Culture Poll November 2018 About the poll The 2018 Fall Cannabis Culture Poll explores attitudes and views on cannabis to better understand the lifestyle of modern cannabis consumers and the future
More informationMULTIPLE LINEAR REGRESSION 24.1 INTRODUCTION AND OBJECTIVES OBJECTIVES
24 MULTIPLE LINEAR REGRESSION 24.1 INTRODUCTION AND OBJECTIVES In the previous chapter, simple linear regression was used when you have one independent variable and one dependent variable. This chapter
More informationA Cooperative Multiagent Architecture for Turkish Sign Tutors
A Cooperative Multiagent Architecture for Turkish Sign Tutors İlker Yıldırım Department of Computer Engineering Boğaziçi University Bebek, 34342, Istanbul, Turkey ilker.yildirim@boun.edu.tr 1 Introduction
More informationAGENT-BASED SYSTEMS. What is an agent? ROBOTICS AND AUTONOMOUS SYSTEMS. Today. that environment in order to meet its delegated objectives.
ROBOTICS AND AUTONOMOUS SYSTEMS Simon Parsons Department of Computer Science University of Liverpool LECTURE 16 comp329-2013-parsons-lect16 2/44 Today We will start on the second part of the course Autonomous
More informationSuggest or Sway? Effects of Online Recommendations on Consumers Willingness to Pay
Suggest or Sway? Effects of Online Recommendations on Consumers Willingness to Pay Gediminas Adomavicius 1, Jesse C. Bockstedt 2, Shawn P. Curley 1, Jingjing Zhang 3 gedas@umn.edu, bockstedt@email.arizona.edu,
More informationA Scoring Policy for Simulated Soccer Agents Using Reinforcement Learning
A Scoring Policy for Simulated Soccer Agents Using Reinforcement Learning Azam Rabiee Computer Science and Engineering Isfahan University, Isfahan, Iran azamrabiei@yahoo.com Nasser Ghasem-Aghaee Computer
More informationEfficiency and design optimization
Efficiency and design optimization Tuesday, Lecture 2 Jeanette Mumford University of Wisconsin - Madison Thanks to Tom Liu for letting me use some of his slides! What determines your statistical power?
More informationUnderstanding Effects of Personalized vs. Aggregate Ratings on User Preferences
Understanding Effects of Personalized vs. Aggregate Ratings on User Preferences Gediminas Adomavicius University of Minnesota Minneapolis, MN gedas@umn.edu Jesse Bockstedt Emory University Atlanta, GA
More informationMeasuring Focused Attention Using Fixation Inner-Density
Measuring Focused Attention Using Fixation Inner-Density Wen Liu, Mina Shojaeizadeh, Soussan Djamasbi, Andrew C. Trapp User Experience & Decision Making Research Laboratory, Worcester Polytechnic Institute
More informationPredicting Heart Attack using Fuzzy C Means Clustering Algorithm
Predicting Heart Attack using Fuzzy C Means Clustering Algorithm Dr. G. Rasitha Banu MCA., M.Phil., Ph.D., Assistant Professor,Dept of HIM&HIT,Jazan University, Jazan, Saudi Arabia. J.H.BOUSAL JAMALA MCA.,M.Phil.,
More informationRecognizing Scenes by Simulating Implied Social Interaction Networks
Recognizing Scenes by Simulating Implied Social Interaction Networks MaryAnne Fields and Craig Lennon Army Research Laboratory, Aberdeen, MD, USA Christian Lebiere and Michael Martin Carnegie Mellon University,
More informationCocktail Preference Prediction
Cocktail Preference Prediction Linus Meyer-Teruel, 1 Michael Parrott 1 1 Department of Computer Science, Stanford University, In this paper we approach the problem of rating prediction on data from a number
More informationA Trade-off Between Number of Impressions and Number of Interaction Attempts
A Trade-off Between Number of Impressions and Number of Interaction Attempts Jacob A. Hasselgren, Stephen J. Elliott, and Jue Gue, Member, IEEE Abstract--The amount of time taken to enroll or collect data
More informationPositive and Unlabeled Relational Classification through Label Frequency Estimation
Positive and Unlabeled Relational Classification through Label Frequency Estimation Jessa Bekker and Jesse Davis Computer Science Department, KU Leuven, Belgium firstname.lastname@cs.kuleuven.be Abstract.
More informationValence-arousal evaluation using physiological signals in an emotion recall paradigm. CHANEL, Guillaume, ANSARI ASL, Karim, PUN, Thierry.
Proceedings Chapter Valence-arousal evaluation using physiological signals in an emotion recall paradigm CHANEL, Guillaume, ANSARI ASL, Karim, PUN, Thierry Abstract The work presented in this paper aims
More informationOn the Use of Opinionated Explanations to Rank and Justify Recommendations
On the Use of Opinionated Explanations to Rank and Justify Recommendations Khalil Muhammad, Aonghus Lawlor, Barry Smyth Insight Centre for Data Analytics University College Dublin Belfield, Dublin 4, Ireland
More informationCHAPTER 3 DATA ANALYSIS: DESCRIBING DATA
Data Analysis: Describing Data CHAPTER 3 DATA ANALYSIS: DESCRIBING DATA In the analysis process, the researcher tries to evaluate the data collected both from written documents and from other sources such
More informationMCAS Equating Research Report: An Investigation of FCIP-1, FCIP-2, and Stocking and. Lord Equating Methods 1,2
MCAS Equating Research Report: An Investigation of FCIP-1, FCIP-2, and Stocking and Lord Equating Methods 1,2 Lisa A. Keller, Ronald K. Hambleton, Pauline Parker, Jenna Copella University of Massachusetts
More informationCase-based reasoning using electronic health records efficiently identifies eligible patients for clinical trials
Case-based reasoning using electronic health records efficiently identifies eligible patients for clinical trials Riccardo Miotto and Chunhua Weng Department of Biomedical Informatics Columbia University,
More informationModeling Uncertainty Driven Curiosity for Social Recommendation
Modeling Uncertainty Driven Curiosity for Social Recommendation Qiong Wu, Siyuan Liu and Chunyan Miao Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly Nanyang Technological
More informationGeneralization and Theory-Building in Software Engineering Research
Generalization and Theory-Building in Software Engineering Research Magne Jørgensen, Dag Sjøberg Simula Research Laboratory {magne.jorgensen, dagsj}@simula.no Abstract The main purpose of this paper is
More informationE-MRS: Emotion-based Movie Recommender System. Ai Thanh Ho, Ilusca L. L. Menezes, and Yousra Tagmouti
E-MRS: Emotion-based Movie Recommender System Ai Thanh Ho, Ilusca L. L. Menezes, and Yousra Tagmouti IFT 6251 11/12/2006 Outline Introduction and Future Work 2 Introduction Electronic commerce Large amount
More informationHow should I explain? A comparison of different explanation types for recommender systems
How should I explain? A comparison of different explanation types for recommender systems Fatih Gedikli a, Dietmar Jannach a, Mouzhi Ge b a TU Dortmund, 44221 Dortmund, Germany b Bundeswehr University
More informationMeasuring the Effects of Interruptions on Task Performance in the User Interface
Measuring the Effects of Interruptions on Task Performance in the User Interface Brian P. Bailey, Joseph A. Konstan, and John V. Carlis University of Minnesota Department of Computer Science and Engineering
More informationReliability, validity, and all that jazz
Reliability, validity, and all that jazz Dylan Wiliam King s College London Introduction No measuring instrument is perfect. The most obvious problems relate to reliability. If we use a thermometer to
More informationHandling Partial Preferences in the Belief AHP Method: Application to Life Cycle Assessment
Handling Partial Preferences in the Belief AHP Method: Application to Life Cycle Assessment Amel Ennaceur 1, Zied Elouedi 1, and Eric Lefevre 2 1 University of Tunis, Institut Supérieur de Gestion de Tunis,
More information5 $3 billion per disease
$3 billion per disease Chapter at a glance Our aim is to set a market size large enough to attract serious commercial investment from several pharmaceutical companies that see technological opportunites,
More informationA scored AUC Metric for Classifier Evaluation and Selection
A scored AUC Metric for Classifier Evaluation and Selection Shaomin Wu SHAOMIN.WU@READING.AC.UK School of Construction Management and Engineering, The University of Reading, Reading RG6 6AW, UK Peter Flach
More informationA Brief Introduction to Bayesian Statistics
A Brief Introduction to Statistics David Kaplan Department of Educational Psychology Methods for Social Policy Research and, Washington, DC 2017 1 / 37 The Reverend Thomas Bayes, 1701 1761 2 / 37 Pierre-Simon
More informationUsing threat image projection data for assessing individual screener performance
Safety and Security Engineering 417 Using threat image projection data for assessing individual screener performance F. Hofer & A. Schwaninger Department of Psychology, University of Zurich, Switzerland
More informationComparative Study of K-means, Gaussian Mixture Model, Fuzzy C-means algorithms for Brain Tumor Segmentation
Comparative Study of K-means, Gaussian Mixture Model, Fuzzy C-means algorithms for Brain Tumor Segmentation U. Baid 1, S. Talbar 2 and S. Talbar 1 1 Department of E&TC Engineering, Shri Guru Gobind Singhji
More informationCHAPTER 2. MEASURING AND DESCRIBING VARIABLES
4 Chapter 2 CHAPTER 2. MEASURING AND DESCRIBING VARIABLES 1. A. Age: name/interval; military dictatorship: value/nominal; strongly oppose: value/ ordinal; election year: name/interval; 62 percent: value/interval;
More informationAssignment 4: True or Quasi-Experiment
Assignment 4: True or Quasi-Experiment Objectives: After completing this assignment, you will be able to Evaluate when you must use an experiment to answer a research question Develop statistical hypotheses
More informationDo Recommender Systems Manipulate Consumer Preferences? A Study of Anchoring Effects
Do Recommender Systems Manipulate Consumer Preferences? A Study of Anchoring Effects Gediminas Adomavicius 1, Jesse Bockstedt 2, Shawn Curley 1, Jingjing Zhang 1 1 Information and Decision Sciences Carlson
More informationUSING AUDITORY SALIENCY TO UNDERSTAND COMPLEX AUDITORY SCENES
USING AUDITORY SALIENCY TO UNDERSTAND COMPLEX AUDITORY SCENES Varinthira Duangudom and David V Anderson School of Electrical and Computer Engineering, Georgia Institute of Technology Atlanta, GA 30332
More informationInvestigating Explanations to Justify Choice
Investigating Explanations to Justify Choice Ingrid Nunes 1,2, Simon Miles 2, Michael Luck 2, and Carlos J.P. de Lucena 1 1 Pontifical Catholic University of Rio de Janeiro - Rio de Janeiro, Brazil {ionunes,lucena}@inf.puc-rio.br
More informationA Cue Imputation Bayesian Model of Information Aggregation
A Cue Imputation Bayesian Model of Information Aggregation Jennifer S. Trueblood, George Kachergis, and John K. Kruschke {jstruebl, gkacherg, kruschke}@indiana.edu Cognitive Science Program, 819 Eigenmann,
More informationRELYING ON TRUST TO FIND RELIABLE INFORMATION. Keywords Reputation; recommendation; trust; information retrieval; open distributed systems.
Abstract RELYING ON TRUST TO FIND RELIABLE INFORMATION Alfarez Abdul-Rahman and Stephen Hailes Department of Computer Science, University College London Gower Street, London WC1E 6BT, United Kingdom E-mail:
More informationOn the Use of Opinionated Explanations to Rank and Justify Recommendations
Proceedings of the Twenty-Ninth International Florida Artificial Intelligence Research Society Conference On the Use of Opinionated Explanations to Rank and Justify Recommendations Khalil Muhammad, Aonghus
More informationNon-Stationary Phenomenon in Large Team Information Sharing
University of Nebraska - Lincoln DigitalCommons@University of Nebraska - Lincoln CSE Technical reports Computer Science and Engineering, Department of 2013 Non-Stationary Phenomenon in Large Team Information
More informationThe Open Access Institutional Repository at Robert Gordon University
OpenAIR@RGU The Open Access Institutional Repository at Robert Gordon University http://openair.rgu.ac.uk This is an author produced version of a paper published in Intelligent Data Engineering and Automated
More informationExploring Experiential Learning: Simulations and Experiential Exercises, Volume 5, 1978 THE USE OF PROGRAM BAYAUD IN THE TEACHING OF AUDIT SAMPLING
THE USE OF PROGRAM BAYAUD IN THE TEACHING OF AUDIT SAMPLING James W. Gentry, Kansas State University Mary H. Bonczkowski, Kansas State University Charles W. Caldwell, Kansas State University INTRODUCTION
More informationPositive and Unlabeled Relational Classification through Label Frequency Estimation
Positive and Unlabeled Relational Classification through Label Frequency Estimation Jessa Bekker and Jesse Davis Computer Science Department, KU Leuven, Belgium firstname.lastname@cs.kuleuven.be Abstract.
More informationDecisions and Dependence in Influence Diagrams
JMLR: Workshop and Conference Proceedings vol 52, 462-473, 2016 PGM 2016 Decisions and Dependence in Influence Diagrams Ross D. hachter Department of Management cience and Engineering tanford University
More informationTITLE: A Data-Driven Approach to Patient Risk Stratification for Acute Respiratory Distress Syndrome (ARDS)
TITLE: A Data-Driven Approach to Patient Risk Stratification for Acute Respiratory Distress Syndrome (ARDS) AUTHORS: Tejas Prahlad INTRODUCTION Acute Respiratory Distress Syndrome (ARDS) is a condition
More informationType II Fuzzy Possibilistic C-Mean Clustering
IFSA-EUSFLAT Type II Fuzzy Possibilistic C-Mean Clustering M.H. Fazel Zarandi, M. Zarinbal, I.B. Turksen, Department of Industrial Engineering, Amirkabir University of Technology, P.O. Box -, Tehran, Iran
More informationControlled Experiments
CHARM Choosing Human-Computer Interaction (HCI) Appropriate Research Methods Controlled Experiments Liz Atwater Department of Psychology Human Factors/Applied Cognition George Mason University lizatwater@hotmail.com
More informationPosition Paper: How Certain is Recommended Trust-Information?
Position Paper: How Certain is Recommended Trust-Information? Uwe Roth University of Luxembourg FSTC Campus Kirchberg 6, rue Richard Coudenhove-Kalergi L-1359 Luxembourg uwe.roth@uni.lu ABSTRACT Nowadays
More informationPRECISION, CONFIDENCE, AND SAMPLE SIZE IN THE QUANTIFICATION OF AVIAN FORAGING BEHAVIOR
Studies in Avian Biology No. 13:193-198, 1990. PRECISION, CONFIDENCE, AND SAMPLE SIZE IN THE QUANTIFICATION OF AVIAN FORAGING BEHAVIOR LISA J. PETIT, DANIEL R. PETIT, AND KIMBERLY G. SMITH Abstract. We
More informationSpatial Cognition for Mobile Robots: A Hierarchical Probabilistic Concept- Oriented Representation of Space
Spatial Cognition for Mobile Robots: A Hierarchical Probabilistic Concept- Oriented Representation of Space Shrihari Vasudevan Advisor: Prof. Dr. Roland Siegwart Autonomous Systems Lab, ETH Zurich, Switzerland.
More informationChapter 20: Test Administration and Interpretation
Chapter 20: Test Administration and Interpretation Thought Questions Why should a needs analysis consider both the individual and the demands of the sport? Should test scores be shared with a team, or
More informationLANGUAGE TEST RELIABILITY On defining reliability Sources of unreliability Methods of estimating reliability Standard error of measurement Factors
LANGUAGE TEST RELIABILITY On defining reliability Sources of unreliability Methods of estimating reliability Standard error of measurement Factors affecting reliability ON DEFINING RELIABILITY Non-technical
More informationSUPPLEMENTAL MATERIAL
1 SUPPLEMENTAL MATERIAL Response time and signal detection time distributions SM Fig. 1. Correct response time (thick solid green curve) and error response time densities (dashed red curve), averaged across
More informationFinding Information Sources by Model Sharing in Open Multi-Agent Systems 1
Finding Information Sources by Model Sharing in Open Multi-Agent Systems Jisun Park, K. Suzanne Barber The Laboratory for Intelligent Processes and Systems The University of Texas at Austin 20 E. 24 th
More informationIf you liked Herlocker et al. s explanations paper, then you might like this paper too
If you liked Herlocker et al. s explanations paper, then you might like this paper too Derek Bridge Kevin Dunleavy Insight Centre for Data Analytics University College Cork, Ireland School of Computer
More informationAbstracts. 2. Sittichai Sukreep, King Mongkut's University of Technology Thonburi (KMUTT) Time: 10:30-11:00
The 2nd Joint Seminar on Computational Intelligence by IEEE Computational Intelligence Society Thailand Chapter Thursday 23 rd February 2017 School of Information Technology, King Mongkut's University
More informationEfficient AUC Optimization for Information Ranking Applications
Efficient AUC Optimization for Information Ranking Applications Sean J. Welleck IBM, USA swelleck@us.ibm.com Abstract. Adequate evaluation of an information retrieval system to estimate future performance
More information