I don't know what the average temperature is, but the weather channel suggests that February is the coolest month, at least in Nassau. It may be different for islands more south. You should check out the weather channel website.
I went to Grand Bahama in February a few years ago and I was freezing! This is coming from a person who lives in central Florida. over 6 years ago