Heat oven to 350°. Line 18 indents in two cupcake pans with paper or foil liners. In a bowl, whisk flour, cocoa powder, baking powder and salt.
Beat butter and sugar in large bowl for 2 minutes, until light colored and smooth. Add eggs, one at a time, beating well after each. On low speed, add flour mixture, alternately with milk. Divide batter among liners, a scant 1/3 cup in each.
Bake at 350° for about 19 minutes or until toothpick inserted into centers comes out clean. Remove cupcakes to a rack; cool completely.
Beat confectioners' sugar, butter and milk in medium-size bowl until smooth. Beat in vanilla. Tint with desired food colors. Spread about 2 tablespoons frosting over each cupcake.