Preheat an oven to 350 degrees F (175 degrees C). Grease or line 24 cupcake or muffin cups with paper baking liners.
Stir cake mix, water, oil and eggs in large bowl until moistened. Beat with a mixer on medium speed for 2 minutes or whisk by hand for 2 minutes. Drop 2 tablespoons of batter into each cupcake cup. Bake in the preheated oven until a toothpick inserted into the cake comes clean, about 8 to 10 minutes. When the cupcakes have cooled completely, cover and freeze until hard, about 4 hours.
Spoon about 1/2 cup of the softened ice cream into each cup and gently pack it into the cup. Cover and return to the freezer until frozen, about 4 hours. Frost the cupcakes with the prepared frosting and serve.