They only move when you are not looking. Light the five fires; a lit fire keeps them out. The cannon fells them, but the noise wakes the rest.