Meta Hierarchical Reinforced Learning to Rank for Recommendation: A Comprehensive Study in MOOCs

Abstract

The rapid development of Massive Open Online Courses (MOOCs) surges the needs of advanced models for personalized online education. Existing solutions successfully recommend MOOCs courses via deep learning models, they however generate weak “course embeddings” with original profiles, which contain noisy and few enrolled courses. On the other hand, existing algorithms provide the recommendation list according to the score of each course while ignoring the personalized demands of learners. To tackle the above challenges, we propose a M eta hierarchical R einforced L earning t o r ank approach MRLtr , which consists of a Meta Hierarchical Reinforcement Learning pre-trained mechanism and a gradient boosting ranking method to provide accurate and personalized MOOCs courses recommendation. Specifically, the end-to-end pre-training mechanism combines a user profile reviser and a meta embedding generator to provide course embedding representation enhancement for the recommendation task. Furthermore, the downstream ranking method adopts a LightGBM-based ranking regressor to promote the order quality with gradient boosting. We deploy MRLtr on a real-world MOOCs education platform and evaluate it with a large number of baseline models. The results show that MRLtr could achieve $\varDelta NDCG_{4}$ Δ N D C G 4 = 7.74%–16.36%, compared to baselines. Also, we conduct a 7-day A/B test using the realistic traffic of Shanghai Jiao Tong University MOOCs, where we can still observe significant improvement in real-world applications. MRLtr performs consistently both in online and offline experiments.