I don't think its all that strange, it wasn't a few years, it was more than 10 years of virtually no play. Even now, I only play 10-15 rounds per year, when I was a 1, I played over 100 rounds a year.
I stopped keeping a handicap altogether during that time. So it wasn't as though I had old scores that were going to be used. I started playing again, spraying balls all over the place and having a tough time breaking 90. So the handicap didn't really go from 1 to 13, it really just started at 13 and now has moved down to 10 after 3 revisions. Still I don't play enough to get it to move all that quickly. My last 20 rounds still date back to 2011. It will keep creeping down, stay tuned. I actually think my ball striking (save for driver) is as good as its ever been. It's a lot closer than the 10 would suggest.
A couple other limiters, I live in downtown chicago, no easy access to a real practice facility. I also do not have a home track, so I frequently play different courses and in many cases courses I have never played. Oh and I also now have an 8 month old.