基于循环神经网络的序列推荐 吴书智能感知与计算研究中心模式识别国家重点实验室中国科学院自动化研究所. Spe 17, 中国科学院自动化研究所 Institute of Automation Chinese Academy of Sciences

Transcription

1 模式识别国家重点实验室 National Lab of Pattern Recognition 中国科学院自动化研究所 Institute of Automation Chinese Academy of Sciences 基于循环神经网络的序列推荐 吴书智能感知与计算研究中心模式识别国家重点实验室中国科学院自动化研究所 Spe 17, 2017

2 报告内容 Recurrent Basket Model (DREAM) Context-Aware Recurrent Neural Networks (CA-RNN) Spatial Temporal Recurrent Neural Networks (ST-RNN) Time-Aware Recurrent Log-BiLinear (TA-RLBL)

3 DREAM

4 Next-basket Recommendation REcurrent basket Model (DREAM) Feng Yu, Qiang Liu, Shu Wu, Liang Wang, Tieniu Tan. A Dynamic Recurrent Model for Next Basket Recommendation. In SIGIR, pp , 2016.

5 Markov Chain Based Methods MC based methods are built based a strong independent assumption.

6 Introduction Conventional models Treat users general interests and sequential behaviors as two divided matters, and combine them in some way; The assumption of Markov Chains only capture local sequential features between two adjacent baskets. Our improvements Incorporate basket representation (local) and sequential behaviors (global) into a recurrent architecture Different pooling methods for dynamic representation

7 Framework of DREAM Pooling operation on the items in a basket to get the basket representation; The input layer comprises a series of basket representations of a user; Dynamic representation of the user can be obtained in the hidden layer; The output layer shows scores of this user towards all items.

8 Experimental Results Datasets: Ta-Feng and T-mall. Compared methods: HRM, FPMC, NMF, MC, TOP. Max-pooling and Average-pooling

9 CA-RNN

10 RNN for Next-basket Recommendation Feng Yu et al., A Dynamic Recurrent Model for Next Basket Recommendation, SIGIR 2016.

11 GRU for Session-based Recommendation Balázs Hidasi et al., Session-based Recommendations with Recurrent Neural Networks, ICLR 2016.

12 Context-awareness (Time) (Location) School: Home: An Inconvenient Truth The Lord Of The Rings Weekdays: Data Mining Weekends: Gone With the Wind Child: Finding Nemo Girlfriend: Titanic Happy: The Merchant of Venice Sad: Hamlet (Companion)

13 Input and Transition Contexts Input Contexts external contexts under which users conduct behaviors; location (home or working place); time (weekdays or weekends, morning or evening); weather (sunny or rainy); Etc. Transition Contexts contexts of transitions between two adjacent input elements in historical sequences; time intervals between adjacent behaviors; capturing context-adaptive transition effects from past behaviors to future behaviors with different time intervals.

14 Sequence with Contextual Information Qiang Liu, Shu Wu, Diyi Wang, Zhaokang Li, Liang Wang. Context-aware Sequential Recommendation. In ICDM, 2016.

15 Context-Aware Recurrent Neural Networks (CA-RNN) Modeling Input Contexts: Modeling Transition Contexts: Context-aware Prediction:

16 Experiments Taobao 13,611,038 shopping records; 1,103,702 users; 462,008 items; input contexts: seven days in a week, three ten-day time periods in a month, and holiday or not; transition contexts: one-day time bins. Movielens-1M 1,000,209 rating records; 3,900 movies; 6,040 users; input contexts: seven days in a week, and twenty-four hours in a day; transition contexts: one-day time bins.

17 Input Contexts VS. Transition Contexts dataset Taobao Movielens contexts MAP NDCG Input Transition all Input Transition all

18 Performance Comparison dataset Taobao Moviele ns method MAP NDC G BPR FM HRM RNN CA-RNN BPR FM HRM RNN CA-RNN

19 Gated Recurrent Units (GRU) GRU VS. RNN: dealing with the vanishing or exploding gradients problem; better handling longer sequences. CA-GRU VS. CA-RNN: context-aware gates; vectors instead of matrices, less parameters.

20 Context-Aware Gated Recurrent Units (CA-GRU)

21 Performance Comparison dataset Taobao Moviele ns method MAP NDCG RNN GRU CA-RNN CA-GRU RNN GRU CA-RNN CA-GRU

22 ST-RNN

23 Public Event Prediction For social good, it is important to predict where and when public events, e.g., terrorist attacts, will happen.

24 Check In Prediction For commercial profit, it is important to predict where users will go next, and recommendation can be made.

25 Recurrent Neural Networks RNN can not well model short-term context for predicting behaviors. Time difference and geographical distance are not considered.

26 Spatial Temporal Recurrent Neural Networks Qiang Liu, Shu Wu, Liang Wang, Tieniu Tan. Predicting the Next Location: A Recurrent Model with Spatial and Temporal Contexts. In AAAI, 2016.

27 Model Learning Prediction: Maximization: Objective: 27

28 Experiments Dataset: Global Terrorism Database (GTD) Gowalla dataset Foursquare dataset Evaluation Metrics: Mean Average Precision (MAP) Area Under Curve (AUC)

29 Experiments

30 Experiments Performance of ST-RNN with varying window width

31 Applications We developed a system on public security events prediction: Situation-Aware Public Security Evaluation System (SAPE)

32 TA-RLBL

33 Multi-behavioral Sequential Prediction App usage prediction A user s behaviors towards apps in an hour, including downloading, using and uninstalling. what app the user is going to download or use next. Multiple types of behaviors Short-term interests Qiang Liu, Shu Wu, Liang Wang. Multi-behavioral Sequential Prediction with Recurrent Logbilinear Model. IEEE Transactions on Knowledge and Data Engineering (TKDE), 2017.

34 Introduction Difficult in modeling sequences with multiple types of behaviors RNN assumes that the temporal dependency changes monotonously along with positions in a sequence, one element usually has more significant effect than the previous one for prediction. For behavior prediction tasks, this assumption does not confirm to complex real situations, especially for the most recent elements in historical sequences

35 RLBL and TA-RLBL Recurrent Neural Networks (RNN) Log-BiLinear (LBL) model Recurrent Log-BiLinear (RLBL) Time-Aware Recurrent Log-BiLinear (TA-RLBL)

36 Experimental Results Experimental summarization

37 Experimental Results

38 Multiple behaviors and single behavior Comparison of multiple behaviors and single behavior

39 Computation time, window width, history length Performance comparison on the Movielens dataset with varying dimensionality and window width Performance comparison with different behavioral history length

40 Thank You! Contact: