Mildenhall is an old market town, mentioned in the Doomsday Book. The town itself has lots of history, and the museum is worth a visit - although, sadly, the Mildenhall Treasure display is a replica, with the original in the British Museum in London. There is a traditional market held every Friday. The parish church of St Marys is worth a visit
Also in the area you have:
Bury St Edmunds
West Stow Country Park and Anglo-Saxon village about 5 years ago
RAF Bassingbourn museum is fantastic! It's just south of Cambridge, 45 min from the base. It was the home of the Memphis Belle during the war! http://www.towermuseumbassingbourn.co.uk/openingtimes.html
It's just down the road from the Duxford Air Museum also!
http://www.iwm.org.uk/visits/iwm-duxford about 5 years ago