Dissolve yeast in the warm milk in a small bowl; allow to stand for a few minutes until it becomes foamy. Whisk together the butter, 2 eggs, salt, 2 teaspoons sugar, and yeast mixture until the sugar has dissolved. Stir in flour to form a smooth dough. Cover with a cloth and allow to rise in a warm place until doubled in bulk, about 30 minutes.
Meanwhile, whisk together 1 cup of milk with the cornstarch in a small saucepan. Cook over medium heat until very hot and steaming, but not quite simmering. Whisk 2 eggs together with 2 tablespoons sugar in a bowl until well beaten. Slowly whisk in half of the hot milk mixture, adding it about 2 tablespoons at a time until incorporated. Stir the egg mixture into the saucepan of milk, and cook gently until thickens and begins to bubble. Remove from heat and stir in vanilla extract. Set the filling aside to cool.
Preheat oven to 425 degrees F (220 degrees C), and line 2 baking sheets with parchment paper.
Punch down the dough, and divide it into two pieces. Roll each piece into a 12x12 inch square, and cut each square into 9 equal pieces about 4 inches on a side. Place about 1 tablespoon of the filling into the center of each piece, and fold a corner across over the filling. Fold the opposite corner across and seal the corners over the filling. Fold the two remaining corners across the filling and seal to make a tidy packet enclosing the filling.
Place the filled buns, folded sides down, on the prepared baking sheets, and brush each bun with beaten egg. Bake in the preheated oven for 15 minutes, until the buns are golden brown, and let cool.
Stir together the confectioners' sugar, hot water, and cocoa powder in a bowl, and spread the icing onto each bun. Refrigerate leftover buns.
