Preheat a skillet over medium heat. Lightly brush one side of each slice of bread with the olive oil. Place 1 slice of bread, olive oil side down into the skillet, and arrange the apple slices evenly over the top. Sprinkle the Swiss cheese over the apple, then top with the remaining slice of bread, olive oil-side up. Cook until the bread is golden brown, then flip the sandwich over, and cook until the other side is golden brown and the cheese has melted, 1 to 2 more minutes.
