There is a contrast made in our passage. This contrast is not between pleasure and pain; rather, the contrast is between one kind of pleasure ("your pleasure") and another kind of pleasure ("delight in the Lord"). Why is God calling His people to turn away from their pleasure? Is God saying that there is something wrong with all of our pleasures? Or is God simply speaking about our sinful pleasures?
When we read the phrase, "doing your pleasure on my holy day," we can see its similarity with the wording of the Fourth Commandment: "Six days you shall labor, and do all your work" (Ex. 20:9). Of particular interest to us is the parallel between "your pleasure" and "your work." You see, the kind of work that the Sabbath command forbade was the work they did for their livelihood. Was it sinful? No, it was the work that God Himself had given them to do, in the promised land! Not to engage in their work with diligence during the other days would be condemned as idleness! And yet the Sabbath command forbade the people of Israel from doing that kind of good and honest work!
So then, "your pleasure" in v. 13 does not just refer to sinful pleasures. It includes the innocent, good pleasures that God Himself gave us to enjoy during the other days! Is God the greatest Killjoy? No! Consider the creation week. The first six days were "good" days. Each of them was punctuated with the divine approval, "God saw that it was good." But what happened on the seventh day? The Lord set it apart ("God… made it holy") and blessed it above the other days (Gen. 2:3)! By way of analogy, we can see that, if God calls us to turn away from our pleasures that are legitimate and good, it can only be that He desires to grant us pleasures that are greater and better on the Sabbath day.
Better in what sense? One way to distinguish the Sabbath rest--delighting in the Lord--from our pleasure can be put in this way: the Sabbath rest is characterized by our delight in the Lord God Himself; our pleasure on the other days is characterized by our delight in the gifts of God. (The two are not mutually exclusive, of course. We are to delight in God all the time. But the Sabbath day emphasizes this more clearly.) And after the resurrection of Jesus Christ, on the Lord's day, our delight in God takes on the heavenly-minded character all the more.