In a medium bowl, combine the cream cheese, mayonnaise, onions, pepper and nutmeg. Stir in the Swiss cheese. Transfer to small ovenproof glass bowl. Sprinkle 1/2 of the almonds on top.
Bake for 7 minutes. Stir, and bake for an additional 8 minutes. Sprinkle remaining almonds on top, and serve.