While ‘X Factor‘ fans will surely miss Paula Abdul and Nicole Scherzinger, their exits may be blessings in disguise. Rumor has it that Simon Cowell has his eye on two record setting pop divas to join his panel: Katy Perry and Mariah Carey!

Following the news of Abdul, Scherzinger and host Steve Jones’ departures, massive speculation arose as to who will fill their respective roles. The loudest whispers have been about Perry (who was a guest judge (and outspoken champion for runner up Adam Lambert) during Cowell’s time at ‘American Idol‘), and Carey.

Cowell has already expressed tremendous interest in attaching Carey to his pet project, a feeling she’s reciprocated. Cowell was determined to find a role for Carey on the first season of ‘X Factor,’ but fate had other plans: When it was decided that she would be his guest judge during the judges’ home auditions, Hurricane Irene grounded her flight. Okay, so maybe it was fate and a paycheck that did it: in December, Carey asked for $300,000 to perform ‘All I Want for Christmas Is You’ with Justin Bieber, but her fee was denied.

Other stars have been thrown into the mix as well. Cowell’s former ‘X Factor’ U.K. judge Cheryl Cole‘s name has made a resurgence in rumors since her replacement Scherzinger left. Perhaps the most interesting, surprising, and admittedly least likely addition is the Harajuku Barbie herself, Nicki Minaj.

While Abdul and Scherzinger each brought their own special charm and chemistry to the show, we can’t wait to see what surprises Cowell has up his sleeves.