Hey, good attempt! Perspective is really tough if your not used it and it does take a while to get used to!
To make it a bit easier on yourself i would keep both points of perspective on the page. If you follow the roofs in your picture the 2 vanishing points will be outside edges of your paper. Remember your vanishing points are a very important reference so while your getting used to drawing in perspective, keep them on the page so you can easily check if your lines are going the right way.
Another thing that I would recommend would be using a set square to get your vertical lines. Its much easier if your working digitally because Photoshop's tools make drawing perfectly horizontal and vertical lines easy, with you using paper and pencil use a ruler and setsquare even if its just to get the main blocks into the picture, but if you your going for a clean finish, you will use these tools for most of the lines.
If it helps as well, I'd try working from reference, get an image from google or take a photo of a street corner your familiar with and go from that, that way you can see how perspective is working in the real world and get a better understanding of how shapes change when looking at different perspectives.
keep it up! it will click eventually!