Come for the idyllic beaches, stay for the afternoon tea, lush botanical gardens and laid-back sophistication.