Heat oven to 180 degrees F. With cooking spray, lightly coat heavy, flat baking sheet and large knife. Place baking sheet in oven.
In heavy 1-quart saucepan, over high heat, bring sugar, syrup and the water to a boil. Cover saucepan for 3 minutes to dissolve any sugar crystals that form on sides of pan. Uncover; insert candy thermometer. Continue to boil until mixture reaches 310 degrees F on candy thermometer, about 2 minutes.
Remove pan from heat. Let stand 30 seconds. With heat-resistant rubber spatula, stir in extract. If desired, add four drops of food coloring to tint sugar mixture. Pour onto prepared baking sheet, tilting sheet slightly to spread candy mixture into 8 x 6-inch puddle. Let stand 3 minutes or until almost firm.
With prepared knife, cut candy crosswise in 1/2-inch wide strips; halve strips; separate. Place in oven; soften, 1 to 2 minutes.
Remove sheet from oven; working with 1 strip at a time, gently stretch and twist into ribbon icicle, 7 inches long. Place on rack to harden. Repeat with remaining strips. If strips harden, return to oven for 2 minutes; if strips get too soft, let cool until firmer. For additional colors, prepare new batch, tinting as desired. Store wrapped or flat on waxed paper in airtight container at room temperature for up to 3 days. Makes about 24 icicles.