The Grand Hotel at northern side of Taipei is a rather spectacular and unusual hotel to stay in (I did on my first Taiwan visit in 82).
In Taipei itself, very intersting and intricately designed temples - Lung Shan Temple near Snake Alley (Huashi), Pao An Temple, Confucious Temple.
Taipei 101 recommended - not a great-looking building from a distance IMHO, but great ambience up close