I almost never play the timed or timed-mode games. Most friends / family who play casual games prefer the untimed games. They like to leave the computer or respond to an instant message without hitting a pause button, and the very reason they ENJOY these games is that it's "low stress" and relaxing - presumably without the pressure of time limits.
Time limits make it easier to increase the challenge, certainly. But is the casual gamer really playing for the challenge, or for the activity?