Olivier was born in Biloxi, Mississippi, while his father, Simon, played for Mississippi of the East Coast Hockey League, and grew up in Levis, Quebec. The undrafted forward made his NHL debut with the Nashville Predators when he played 8:03 of a 2-1 loss to the Winnipeg Jets at Bridgestone Arena on Nov. 19, 2019, to become the only Mississippi-born player in League history.

Olivier signed a six-year, $18 million contract with the Columbus Blue Jackets on March 5, 2025, nearly three years after he was traded to Columbus by Nashville. He finished that season with 139 penalty minutes and 306 hits, second in the NHL to Nikita Zadorov of the Boston Bruins (145) and Kiefer Sherwood of the Vancouver Canucks, respectively, while breaking Brandon Dubinsky's Blue Jackets record (248, 2016-17) for hits in a single season.

Olivier signed a two-year, entry-level contract with the Predators on May 1, 2019. He's played in the American Hockey League (Milwaukee) and the Quebec Major Junior Hockey League (Sherbrooke, Shawinigan and Moncton). He was an alternate captain for Sherbrooke in 2017-19 and helped the Phoenix to the 2016 President's Cup Final, a five-game loss to Rouyn-Noranda.