Whether you are rewriting your URLs to make them more search engine friendly, rearranging your content or moving everything to a brand new domain, 301 redirects are the way to accomplish a smooth transition.
A 301 redirect is like a change of address card for search engines and browsers, telling them that the URL address they are looking for has permanently moved. Other types of redirects exist, but the 301 redirect is the only one that search engines consistently look upon favorably. They will see the change of address and treat the new page as if it was the old one.
Each search engine stores an index of every page it visits, and it questions the validity and authority of Web sites whose pages disappear without explanation. Links coming into the old URLs suddenly go nowhere. Those inbound links were bringing authority to your site as independent votes of confidence in your site, but if the page disappears so does the vote.
Missing pages bother both the human visitor and the search engine, because they cannot reliably find the information that was once there. Your visitors will likely leave your site when encountering a 404-not-found error. Search engines will see that your site is not delivering a good experience to searchers and will decrease your search result placement. A 301 redirect eliminates these issues.
Inbound links also pass along link juice, which is the amount of Page Rank that passes from the referring page to the linked page. If you have managed to get links from high Page Rank sites, you want to keep as much of that juice flowing as possible.
While you can, and should, build new links to the new page, the age of links also plays a role in how much juice and authority a given link provides. Older links are more valuable than new links, so you want to keep those established links. A 301 redirect keeps the link juice flowing.
Higher search result placement means higher traffic volume, so if the old page showed up in the top of search results a 301 redirect will help the new page to appear at the top. This does not happen immediately, and in the short term your ranking and traffic will suffer.
The search engines do not immediately connect the new page with the old rank. They will first find the new page and treat it as a new page with no authority. They will also notice that the old page with authority no longer exists. In much the same way that it takes time before the post office gets all mail rerouted to a new address, it takes time for the search engines to learn about a page’s new address. Once it does, traffic is restored and you do not have to build traffic from scratch.
In the end, a properly executed 301 redirect is one of the most useful tools you can use in any Web site redesign.