Preheat grill to medium high.
Cast iron skillets and grills are made for each other. Cast iron is the best heat conductor for cooking. And the even heat of the grill makes this potato torte crispy on the outside and tender on the inside.
Generously butter a well-seasoned 10-inch cast iron or other heavy skillet.
Using a V-slicer or mandolin, thinly slice each potato, placing slices in the skillet as you go to prevent oxidation.
Because you will invert the torte after it is cooked, the bottom layer of potatoes will be the top of the torte so make sure to arrange the slices in an attractive pattern.
As you add each layer, brush it with olive oil, and sprinkle with salt and pepper and about 1/2 teaspoon of rosemary.
When you're finished, you should have about 7 layers of potatoes.
Place skillet on the grill, and cook until the potatoes are sizzling nicely, about 12 to 15 minutes.
Using heat-resistant gloves or potholders, remove the skillet from the grill and drain off excess oil. Carefully invert the torte onto a clean plate and then slide the potatoes, bottom side up, back into the skillet.
Return to the grill, and cook, with lid closed, for about 20 to 25 minutes or until potatoes are browned and crispy and inner layers are tender.