The use of incense by God’s people goes at least as far back as the sacrificial system established by God for the nation of Israel. The incense symbolized the sweet aroma of the prayers of God’s people rising up into heaven, which prayers were a delight to God. This same practice and symbolism was a part of the church at least up to the time of the Reformation.
Calvin and those who followed him took the position that the use of incense was a part of the Old Testament ceremonial law, and thus should not be allowed. The book of Hebrews explains how the Old Testament sacrificial system was a shadow, a type, and Christ the reality or anti-type. To go back to the shadow is to implicitly deny that Christ has come once for all. As Paul warned the Hebrews against going back to the shadows, so Calvin warned Protestants against going back.
The English Reformation eventually birthed the Puritans whose desire it was to purify the church of the remains of Roman theology and worship. They vehemently rejected the use of incense, and eventually exercised influence beyond the Church of England. As such most, but not all, Protestant churches reject the use of incense.
While I understand the reasoning of Calvin, the Puritans and those who follow after them I take a different perspective. It is absolutely right and proper that we should never go back to a sacrificial system that pointed us toward the finished work of Christ. His work is finished, and there is no reason for going back. That said, we ought not to reject everything that was established in the Old Covenant. The Psalms were sung in the Old Covenant, but we certainly may and I would argue should still sing them today. God’s people gathered in the Old Covenant, and we do not deny the coming of Christ by gathering together in the New Covenant. We should have a deep fear of reinstituting Old Covenant sacrifice, but not be afraid to repeat any and everything they might have done.
They, and we, for instance, pray. What is it, I wonder, about the glorious reality of the finished work of Christ, that is implicitly denied by continuing to symbolize our prayers ascending in the incense? We still pray. God still delights to hear our prayers. We can be confident that they are indeed still a sweet aroma to Him. So why not the incense?
We ought to reject what became Rome’s false gospel. We ought to reject man-made additions and sundry forms of strange fire. We ought to reject sacrifices. I cannot see, however, how all of these truths mean we may no longer use incense in worship. Indeed I would suggest that if and as we use incense we remember that the faith did not begin with us, with the Puritans, with Calvin, not even with the New Covenant. We remember that we are a part of the one people of God that stretches all the way back to the Garden. That in turn is not just a sweet aroma to God, but a sweet aroma to us as well.