I was there a few days ago. For me, I found it really was about about Alhambra. Walking around the old town is also fantastic. The town is lovely and easy to get around, but for me a day and a half was plenty.
The key piece of advice is book your ticket to the Alhambra!!!! We booked 3-4 weeks in advance and could only get a ticket for a guided tour, everything else was sold out.
Another thing many people do is to visit an arabic bath, hammam. It might sound daunting or weird, but it's quite a pleasant experience. I recommend the full package.
Also, they will give you 10% off if you show them your Alhambra ticket. (Be sure to book this one ahead of time too though)
When I tried to book they were full, instead I went to Aljibe San Miguel. It is looked a bit offputting from the outside or a random backstreet, but it really was lovely!
http://www.aljibesanmiguel.es/ about 3 years ago