Preheat oven to 350 degrees F (175 degrees C). Put garlic in small ovenproof bowl, and drizzle with olive oil. Roast for 30 minutes, until very soft. Cool, peel, and reserve oil.
Cook potatoes and yams in a large pot of salted water until tender, about 20 to 30 minutes. Drain, reserving 1 cup liquid.
Place potatoes in a mixing bowl. Add milk, butter or margarine, rosemary, garlic, and reserved olive oil. Mash until smooth as desired, adding reserved cooking liquid as needed. Mix in 1/4 cup cheese. Salt and pepper to taste. Transfer to a buttered or oiled 8 x 8 x 2 inch baking dish. Sprinkle with remaining cheese.
Bake for 45 minutes, until heated through and golden on top.
