Center a rack in the oven and preheat to 350 (F).
In a medium bowl, whisk together 2 cups all-purpose flour, 1 3/4 teaspoons baking powder, and 1/2 teaspoon kosher salt until well combined. Set aside.
In the bowl of a freestanding electric mixer fitted with a paddle attachment, combine 1/2 cup unsalted butter and 1/2 cup granulated sugar. Beat together on medium-high speed for 2 minutes, until the mixture has doubled in volume and becomes light and fluffy.
Once the mixture is light and fluffy, lower the mixer speed to its slowest setting and add 2 large eggs one at a time, only adding the second egg once the first is fully incorporated. Add 1 teaspoon pure vanilla extract.
Once the vanilla extract is incorporated, keep your mixer on the lowest setting and add half the flour mixture (from the 2nd step). Once the flour is fully incorporated, add 3/4 cup whole milk. Continuing mixing for about a minute or so, just until the mixture looks like it's completely moist, then add the remaining half of the flour mixture. Mix just until the flour streaks disappear into the batter — at this point, the batter will probably look like it's curdled. But don't worry! It's supposed to look that way. It'll come together in the oven, I promise.
Pour the cake batter over the citrus slices in the cake pan, using an offset spatula to spread the batter so it's nice and even across the pan. Transfer to the preheated oven and bake for 40 to 50 minutes, or until the bottom of the cake bounces back when gently poked and a wooden pick inserted in the center comes out clean. Remove the cake from the oven and transfer to a wire rack to cool for 5 to 10 minutes. Don't wait any longer than that — the cake might get stuck in the pan if you let it cool to room temperature completely! After 10 minutes, run a butter knife or an offset spatula around the edge of the cake. Place an upside down plate on top of the cake pan and flip the entire thing over to let the cake fall out onto the plate. Serve slightly warm, with a dollop of soft whipped cream and garnishes of fresh thyme.