Yes, you can, but its not as obvious as one would think.
When you are using the Google Maps website you can easily drop into Street View by zooming all the way into the map or by dragging the Google Pegman to the map. However, as you can see below, when you are viewing an embedded Google Map you don't have these abilities.
To Embed Google Street View, open Google Maps and use Street View to bring up the location you want your embed to start from. Once you are in Street View, click on the Link button in the top right corner and ether copy the embed code or click the customize link if you want to change the size of the embedded map. You can use this code to embed a Street View map using Embed the Web.