It’s probably because you think you need to relax during your free time. Most people divide all time into either work or free time. Scott Young, the author of Why Your Free Time is Boring says:
But the subtle message contained in this split is that work and leisure
shouldn’t resemble each other. Your work needs to be productive,
efficient and challenging. Therefore leisure should be relaxing,
accomplish nothing and be free of pressures.
The problem? We have the most fun when we are most engaged, challenged or using our skills.