Savory spice has an affinity with bay leaves, fennel, rosemary, sage and thyme. Savory has a strong peppery taste. Summer Savory spice is used most often because it’s milder than the winter variety which has a stronger sharper, more bitter taste.