Butter a 9×5–inch loaf pan, or a 4 x 12 ½-inch rectangular terrine, and line with parchment paper, extending the paper over the long sides by a couple of inches. Pre-heat oven to 350° F. Taste the olives to determine salinity and add salt accordingly. Measure the chopped olives and cheese into a small bowl.

In a large bowl using a wire whisk, mix together the flour, sugar, baking powder, baking soda, rosemary, and salt (if needed). Toss about 1/3 cup of the flour mixture into the bowl with the olives and cheese and mix together. This extra step assists in the distribution of these ingredients when they are later folded into the batter.

In another bowl, beat the egg until foamy, then whisk in the oil and buttermilk. Using a rubber spatula, fold the egg mixture to the dry ingredients just until incorporated. Add the olives and cheese, and mix in gently.

Pour into the prepared pan and bake for about 25-30 minutes, or until the top of loaf is browned and a tester inserted into the cake comes out clean.