Thamel, the most touristy part of Kathmandu, is probably the best place to stay as a tourist. It is central and convenient and packed with guesthouses. Otherwise there is Freak Street, next to Durbar Square, but it is not the thriving neighbourhood I believe it once was. over 8 years ago
I would personally recommend Hyatt Hotel right in the heart of Kathmandu. It is a convenient location if you want to experience the best Nepal can offer. The concierge can arrange for speedy transportation to anywhere in Nepal as your heart desires. It is an upmarket hotel unlike the many lodges and rest houses available in Thamel or it's surroundings.
But be warned that it's rate is not exactly cheap as compared to the others mentioned. about 8 years ago