Do I need to clean off thermal paste?
You should remove the old paste before applying new one for proper thermal conduction. Use cotton dipped in isopropyl alcohol to clean old paste.. Liquid metal thermal paste has better conductivity but is more difficult to apply and even more difficult to clean up..
Is dried up thermal paste bad?
With dried up thermal paste heat conductivity is not so good anymore. Basically you're having a thin layer of bad heat conductor (insulator) between cpu and heatsink .