Melt the butter in a large saucepan over a medium heat.
Stir in the brown sugar and milk until melted, and then bring to the boil for 2-3 minutes, without stirring.
Remove from the heat, and stir in the peanut butter.
Add the icing sugar and stir well until there are no lumps.
Transfer into a lined 20cm square baking tray, and set aside to cool to room temperature, then refrigerate until completely set.
Cut fudge into squares with a sharp knife and store in an airtight container.