In a small bowl combine the oil, lemon juice, garlic, oregano, salt and pepper. Mix together. Place chicken in a 9x13 inch baking dish and pour oil mixture over chicken.
Stuff each chicken breast with 1 slice feta cheese and 1 slice bacon. Secure open sides with toothpicks.
Bake uncovered at 350 degrees F (175 degrees C) for 30 to 35 minutes.