Gamasutra - Game Design Essentials: 20 Difficult Games
But this is not to say that games must be easy. The impulse to make video games easier can be traced to a fundamental change in perception over what a game should be. The older school of thought, which dates back and beyond the days of Space Invaders to the era of pinball, is that a game should measure the player's skill. Arcade games, in fact, must make it difficult for a player to last for any great length of time in order to keep money coming into the coin box. The newer concept is that a game should provide an experience to the player. The player is to feel like some character, or like he's participating in a story, or that he's making some difference in a fictional realm.
The difference can be seen in Super Mario Bros., but may date to before it. Super Mario Bros. doesn't seem like it's entirely convinced either; many of the later levels are quite hard. The discarding of score, the emphasis on "suspension of disbelief" and storytelling, the trend towards providing a narrative longer than "see those guys; kill them," and the idea that a game can be "finished," these are all substantial indications of the change. A game intended only to measure skill, particularly, shouldn't have a completion state, because it would be like the game has given up.