Kingsley House was once the Old Rectory, and had a reputation for being haunted by a "fairly benign presence" known as Button Cap, who was known to make scratching noises, to be heard breathing and stroke the heads of children.

 

The poltergeist phenomena was said to occur around 1825, but some regarded the case as "frivolous".