Motion of continents relative to the Earth’s spin axis may be due to either rotation of the entire Earth relative to its spin axis - True Polar Wander (TPW) - or due to motion of individual continents. Here we present a new method to separate between these for the last 320 My, based on the global average of continental motion and rotation through time in a paleomagnetic reference frame. Two main components are identified: a rather steady northward motion, and, during certain time intervals only, clockwise and counterclockwise rotations, which are interpreted as evidence for true polar wander (TPW). We find ~18 degrees counterclockwise rotation between ~ 250 and 220 Ma and the same of amount clockwise between ~195 and 145 Ma. The rotation axis is, in both cases, close to the reconstructed eruption site of the Central Atlantic Magmatic Province (CAMP) at about 200 Ma, which developed into the North American - South American - African triple junction. This is followed by further ~10 degrees clockwise rotation between ~145 and 135 Ma, followed again by the same amount of counterclockwise rotation between ~110 and 100 Ma, with a rotation axis further east, in the reconstructed area of North Africa / Arabia. These rotation axes mark the maxima of the degree two geoid during those time intervals, and the fact that the overall net rotation since 320 Ma is nearly zero is an indication of long-term stability of the degree two geoid and related mantle structure. The latter is quite close to the present-day centers of two Large Low Shear Velocity Provinces in the lowermost mantle, which is a further indication for their long-term stability. For the former, the location further west may reflect the contributions of geoid highs related to subduction at the western edge of Pangea, or be related to the CAMP plume. Cumulative TPW between late Carboniferous and the Neogene has been negligible, whereas continents have moved substantially north relative to the underlying mantle. Northward motion could be caused by mantle upwellings being stronger in the southern hemisphere, corresponding to an average northward component of mantle flow at shallow depths. We propose a new reference frame, mostly based on paleomagnetism, and corrected for the TPW identified in this study, that would be the most appropriate for relating surface to deep mantle processes between 320 and 130 Ma.