There is no rule that you have to stay in one place your whole life. There are a lot of factors that go into where the best place is for you to retire. It may just so happen that the best place is not where you are currently.
Kids are grown and have moved so there is nothing stopping you from going to your dream area. Or, you are now on a serious budget and your area is too expensive to live a good retirement. Then, there are also health concerns and where you are now may not have the right weather or amenities for your condition.