There are many Thai banks around the city, as well as Chinese money changers. Exchange rates from the latter for Malaysian ringgits may be significantly better than in the banks.
ATMs are abundant in Hat Yai, although most of them, as everywhere in Thailand, charge 180 baht fee for foreign card transactions.
Minimarts such as 7-Eleven are ubiquitous and open 24 hours. Good for a quick meal, stocking up food and personal items. Or head to Tops Market for more high-end groceries at Robinson Department Store, Central Department Store, or CentralFestival.
A variety of English-language books can be purchased at Asia Books shop inside the B2S on the second floor of CentralFestival.