Heat olive oil in a skillet over medium heat; cook and stir shallot until translucent, about 5 minutes. Pour balsamic vinegar into skillet and bring to a boil. Reduce heat to low and simmer until balsamic vinegar mixture is reduced by half, about 10 minutes. Stir often.
Cut chicken breasts from one side through the middle horizontally to within one-half inch of the other side. Open the two sides and spread them out like an open book.
Spread half the goat cheese onto one half of each chicken breast and drizzle 1/3 of the reduced balsamic vinegar mixture over the goat cheese. Close the chicken breasts over the goat cheese and secure with toothpicks. Arrange chicken into a baking dish. Drizzle with remaining 1/3 of the balsamic reduction.
Bake in the preheated oven until the chicken is no longer pink inside, the filling is hot, and the juices run clear, 30 to 35 minutes. An instant-read meat thermometer inserted into the center of a filled breast should read at least 160 degrees F (70 degrees C).
