With vegetable peeler, remove 4 strips of zest from lemon. Combine with olives in large bowl. Squeeze enough juice from lemon to equal 1 tablespoon. Add to olives. Stir in chilies, olive oil, fennel, thyme, pepper and bay leaf. Spoon into sterilized, wide-mouth glass jars.
Refrigerate for up to 1 week.