The Colts gave Sheard a three-year deal for $25.5 million, including $12.75 million guaranteed, a source told ESPN.
"Jabaal brings a wealth of experience and veteran leadership to our locker room," Colts general manager Chris Ballard said. "His size and speed are the conduit to his pass-rushing ability as he creates matchup difficulties for opponents. Jabaal is equally talented against the run and in pass coverage. His addition provides a good player for our defense, and we're excited to have him as a member of the Colts."
The 6-foot-3, 265-pound Sheard finished second on the Patriots with five sacks during the 2016 regular season, and he arguably played his best game in Super Bowl LI when he was used in a variety of spots, including dropping into coverage. He had two tackles and a half-sack in the game.
Sheard was projected to start as the top replacement for defensive end Chandler Jones, who was traded to the Arizona Cardinals in March 2016. Sheard was ultimately passed on the depth chart by second-year player Trey Flowers (who had a team-high seven sacks) and settled into a rotational role, which was the same spot in which he had success with the Patriots in 2015 when he had eight sacks.
Sheard's low point in 2016 came on Nov. 20 when he was a healthy scratch for a road win over the 49ers. He rebounded and became a productive contributor down the homestretch.
"Jabaal is a tough football player who can help set the edge against the run and get after the quarterback," Colts coach Chuck Pagano said. "He will be a good addition, and I look forward to seeing him contribute to our team."
Sheard, who turns 28 on May 10, had signed a two-year, $11 million contract as a free agent before the 2015 season. He spent the first four years of his career with the Cleveland Browns, who drafted him in the second round (37th overall) in 2011.
ESPN's Mike Reiss contributed to this report.