Probably developing a flat spot between the brushes and commutator. Pretty common. Get it rebuilt sooner than later as the poor connection will cause a higher draw and more damage.
Yes, do the usual checking of the connections and grounds first.
.....and never rule out the battery just because the lights work. Starter requires far more juice!