Line three cookie sheets with parchment paper; set aside.
Combine flour, baking powder, and salt in a medium bowl. Beat butter, peanut butter, and sugar in a medium bowl with an electric mixer until light and fluffy, about 3 minutes. Add egg and beat until blended. Beat in vanilla extract until smooth, scraping down the sides. Gradually add in flour mixture. Cover and chill dough for 1 hour or until easy to handle.
Preheat the oven to 350 degree F. Shape dough into 3-inch logs. Place logs about 2 inches apart on the prepared cookie sheets. Flatten to 1-inch wide rectangles. Use the handle of a wooden spoon to make three indentations in dough to look like a traffic light. Bake cookies for 4 minutes, remove from oven, and re-press indentations. Bake another 4 to 6 minutes, until edges of cookies are golden and just firm. Transfer to a wire rack and cool completely.
Divide apple jelly between two small bowls and tint each with a different food coloring. Fill cookie indentations with jelly using a small spoon or, for a neater look, pipe them in using resealable plastic bags. Spoon the three colors of jelly into separate reesealable bags. Snip a very small corner from each bag and pipe a dot of jelly into each indentation on the cookies in this order: green, yellow, red, with red at the top.