Preheat oven to 350 degrees F (175 degrees C). Line a baking pan with aluminum foil.
Arrange the mahi mahi fillets in the prepared baking pan. Pour lemon juice over the fillets, and sprinkle with thyme and basil. Top each fillet with pineapple. Pour milk into the pan, not directly on the fish.
Bake 25 minutes in the preheated oven, or until fish flakes easily with a fork.