It was reported in 2018 that Prince Harry and Meghan Markle would be relocating from Nottingham Cottage at Kensington Palace to Frogmore Cottage in Windsor. The need for greater privacy and security, particularly with Archie, influenced Prince Harry and Meghan’s choice to leave London and go to Windsor.

The Sunday Times revealed later that Harry and Meghan originally planned to reside at Windsor Castle alongside Queen Elizabeth and Prince Philip.

Despite the fact that Meghan and Harry later settled into family life (with Archie) at Frogmore Cottage, the couple are said to have had their hearts set on Windsor Castle, which is only a 10-minute drive away, and are said to have asked the Queen if living quarters could be made available after marriage, according to The Sunday Times.

Unfortunately, the plan did not work out, most likely because the Queen determined that the castle was not the ideal location to nurture a young, developing family.

“The couple are understood to have set their hearts at first on Windsor Castle, and are believed to have asked the Queen if living quarters could be made available after their marriage,” says Roya Nikkhah’s article. The Queen, however, rejected that plan, and Frogmore Cottage got a $3.05 million renovation for the Sussexes.

The Queen spends most of her weekends at Windsor and is in formal residence there between Easter and June. It is the world’s biggest inhabited palace, with over 1,000 rooms, including private and state apartments.

“There are empty bedrooms and suites in the private apartments which the Sussexes may have had their eye on, or perhaps some former living quarters in the castle grounds converted into other things,” Hugo Vickers, a royal author and deputy lord lieutenant of Berkshire, told The Times. “But I can see how it might not be entirely appropriate to have a young family living there.”

The Sussexes have now completely renovated Frogmore Cottage, adding soundproofing, a carefully planned nursery, Soho Home touches, royal art, and custom made items. They’ve also welcomed many renowned faces, including George and Amal Clooney, Jane Goodall, Jessica Mulroney, and, of course, a slew of royals, including Queen Elizabeth, Prince Charles, Prince William, and Kate Middleton.

Even though they didn’t reside in the Castle, the town of Windsor proved to be very important to Harry and Meghan.

“As with all members of the Royal Family, Windsor is a very special place for Prince Harry and he and Ms. Markle have regularly spent time there over the last year and a half,” Kensington Palace said in a statement issued before of the Sussexes’ wedding in 2018.

“Prince Harry and Ms. Markle are delighted that the beautiful grounds of Windsor Castle will be where they begin their lives together as a married couple.”

And who knows, maybe Queen Elizabeth will give the Sussexes another house in the near future, similar to the one she granted the Duke and Duchess of Cambridge.

The Duke and Duchess of Sussex are believed to have a special fondness for the location; after all, it was where they celebrated Archie’s baptism earlier in 2019 and where they married in 2018 in the 15th century St George’s Chapel within the castle walls.