Recommendation systems and reinforcement learning concepts

Collaborative filtering recommends items using the past behaviour or ratings of similar users. Content-based filtering recommends items whose descriptions resemble what the user already liked. Hybrid systems combine both. The cold-start problem arises when new users or items have no history, so reliable recommendations are hard.

Reinforcement learning (RL): an agent in a state s takes an action a, receives a reward and moves to a new state. The goal is to maximise cumulative reward. The exploration versus exploitation trade-off balances trying new actions against using the best known one; an agent that always picks the best-looking action and never tries others may never discover a better action.

Q-learning updates Q(s,a) with:

Q(s,a) = Q(s,a) + alpha x [reward + gamma x max Q(s', a') - Q(s,a)]

Worked example: Q = 2, alpha = 0.1, reward = 1, gamma = 0.9, best next value 5. Target = 1 + 0.9 x 5 = 5.5. Difference = 5.5 - 2 = 3.5. New Q = 2 + 0.1 x 3.5 = 2.35.

q = 2 + 0.1 * (1 + 0.9 * 5 - 2)
print(q)

The value 5.5 is the target and 3.5 the error, not the updated value.

Exam trap: collaborative filtering uses other users' behaviour, not item text; and an epsilon of zero means no exploration at all.