The short answer is: a castle (German: Burg) is a defensive and often hard to access fortification until the late Middle Ages (with small windows, low doors and narrow staircases plus towers, walls, drawbridge and moat), a palace is a rather representative building after the 17th century.
The somewhat longer answer has to do with firearm development (especially cannons in the 17th century). As a cannon was able to bring down a castle wall or tower within hours this fact had a dramatic impact on castle architecture since about 1650. There was no point to establish fully fortified compounds and a paradigm shift evolved from defensive fortification to representation. The early palaces retained more defensive functions than the later exclusively representative palaces.
Castles originated from defensive settlements or fortified military bases out of wood (such as “castellum” under the Roman empire). These compounds were effectively fenced in living and utilitarian buildings with one or more towers for viewing or last line of defense.
Both types of compounds served as accommodation of the ruling families in feudal systems and typically were elaborate, large (compared to today’s accommodations) and had multiple additional functions which we do not know anymore today (carcels, stables, powder chamber etc.) or which today are not part of our culture anymore (knight’s hall, ball room, chapel etc.). In regions with a high density of castles (i.e. Loire region in France) neighboring castle owners tried to outdo each other leading to larger and more elaborate castles over time. One extreme case is the Chambord palace (Loire region) with over 400 fire places.
Not infrequently, a palace was built on the foundation of a destroyed castle. Frequently, intact castle elements carried over (central tower or “Burgfried”), a high wall or a moat combined with fixed or draw bridge). As a consequence once will find today not infrequently hybrid architecture (palace with castle elements) and a 1000 year castle history is peppered with numerous destructions and reconstructions.