The First Presbyterian Church of Caledonia, with 68 members of Scottish and Swedish descent, was formed in 1888. On October 26, 1894 an Annual Meeting was held in the new church. The congregation was served by several ministers in the early years but on December 16, 1901, Rev. C.M. Boyce was installed and served, with one six-year interruption, until his death in 1933. The white clapboard church burned to the ground in October 1933. It was replaced by the present Gothic-style building, which was dedicated on September 7, 1936. In 1979, the name of the church was changed to Spruce Presbyterian Church.