The cheapest places are the guesthouses, which cost a few dollars per bunk. The leading guesthouses (which are more like hostels) are The Khongor Guesthouse and UB Guesthouse. If you want a private room, these places have quite small rooms. Somewhere like Zayas has more space but no community.
Cheaper hotels include the more outdated ex-Soviet hotels like the Bayangol. Palace Hotel (south side of the city) is not too expensive for a reasonable standard, but you need to take a taxi into town. about 9 years ago