Whisk eggs, whites, sugar and salt in 2-quart saucepan. Gradually stir in low-fat milk. Cook over low heat, stirring, until mixture thickens and coats back of spoon, about 25 minutes (should register 170°F to 175°F on instant-read thermometer); do not let boil. Pour into bowl. Stir in brandy if using, vanilla and nutmeg. Refrigerate to chill, 3 hours.
Pour skim milk into ice-cube tray. Freeze 2 hours or until almost frozen. Empty cubes into metal bowl. Beat at medium speed until soft peaks form, 10 minutes. Gently fold into custard mixture. Pour into chilled punch bowl. Sprinkle with nutmeg and serve with cinnamon stick if you wish.