1. Heat oven to 350 degrees . Coat a 9-inch square baking pan with nonstick cooking spray. Line bottom with wax paper; coat paper with nonstick cooking spray.
2. In a large bowl, whisk together cake flour, cardamom, cinnamon, ginger, salt, baking powder and baking soda. In a second large bowl, beat egg whites until frothy. Add 1/4 cup of the sugar and beat until stiff, shiny peaks form. Set aside. In a medium bowl, beat egg yolks. Split vanilla bean in half lengthwise and scrape seeds from pod. Add to egg yolks along with 1/2 cup of the sugar. Beat until pale yellow and fluffy, about 2 minutes. On low speed, beat in milk, olive oil and remaining 1/2 cup sugar. Whisk egg yolk mixture into flour mixture.
3. Fold whipped egg whites into batter and pour into prepared pan. Bake at 350 degrees for 55 minutes, until cake springs back when gently pressed.
4. Meanwhile, prepare frosting: Beat together Neufchatel and butter until smooth and no lumps remain. On low speed, beat in confectioners' sugar and salt. Add heavy cream and beat on medium until fluffy and smooth.
5. Cool cake in pan on a wire rack for 10 minutes. Invert cake onto a rack; remove wax paper and turn right-sideup onto rack. Cool completely (about 2 hours) and spread top with frosting. If desired, garnish with dried fruit or lemon peel and pepitas.