It has a complex building history, as can be seen once inside the church, where there are many Saxon features, including tiny triangular windows, and high up on the west wall, a beautiful pair of windows with pointed tops and a carved baluster.
There are several excellent pieces of Saxon sculpture here, thought to date from the 9th century, including an angel in the ruined apse. Especially interesting is the font, rescued from use as a wash tub on a farm, and covered in intricate interleaving carved patterns.
Recently, an early painted figure was discovered on a panel high up in the east wall of the nave. It may date from the 10th century, which would make it the oldest wall painting in any church in Britain.
Close to St Mary's is another Saxon church, called Odda's Chapel, which had been converted into a farmhouse but is accessible.