Center a rack in the oven and preheat to 325 (F).
In a medium bowl, combine 1 cup granulated sugar and the zest from 1 freshly grated lemon. Use your fingers to rub the zest into the sugar — this will help release oils from the lemon zest and make the sugar aromatic. Set aside once the sugar starts to become fragrant.
In a medium bowl, whisk together 2 cups cake flour, 1 1/4 teaspoons kosher salt, and 1 teaspoon ground ginger until well combined. Set aside.
In the bowl of a freestanding electric mixer fitted with a paddle attachment, combine 1 cup granulated sugar and lemon zest (from the 1st step) and 1 cup unsalted butter. Beat on medium-high speed until the mixture is light and fluffy, at least 5 minutes, stopping to scrape down the bowl halfway through. Once the mixture is light and fluffy, slow the mixer down to its lowest setting. Add 4 large eggs, one at a time, only adding the next egg after the previous one has been fully incorporated. Once all the eggs have been added, add 1 teaspoon vanilla extract and continue mixing until fully incorporated, before adding 1/3 cup sour cream and 2 teaspoons lemon juice. Continue mixing until fully incorporated, then stop the mixer and use a rubber spatula to scrape down the sides of the bowl. Turn the mixer back to its lowest speed and add the flour mixture (from the 2nd step), 1/4 cup at a time, until just incorporated. It's okay to stop the mixer a little early if you have 1 or 2 flour streaks left, and finish the rest by hand using a rubber spatula. Transfer the batter to the cake pan containing the brown sugar and rhubarb, spooning it over the fruit. Use an offset spatula to smooth out the top.
Transfer the baking sheet containing the cake pan to the preheated oven and bake for 1 hour and 15 minutes, or until the top of the cake bounces back when gently touched and a cake skewer inserted into the center of the cake comes out without any large, moist crumbs.
Place the pan on a wire rack and cool for 15 minutes. After 15 minutes, run a knife around the cake, place a plate on top of the pan, and turn it upside-down. Tap on the base of the cake pan until the cake releases onto the plate. Once the cake is on the plate, allow to cool on a wire rack. Serve while warm.