The church is a building on the Island which Eko and Charlie were working on until Eko found a new purpose in pushing the button in the Swan. Following Eko's movement into the Swan, Charlie attempts to continue work but finds it difficult and gets frustrated with Eko's abandonment of their project.
Just warning you, that if you haven't watched the whole show might see some information before you are ready. To help with that we have this nifty filter. Just set it to the episode you you are upto, and we'll do our best not to spoil anything.