Your question is awesome bro!, this struggled me many times, and as now I'm happy with the circles I can draw they are still far from being a perfect one.
In my experience, there is no right answer, because there can be many reasons why the circle just don't appear, but the important thing is that simply is there no meant to be. Is not the faq that you can or cannot draw the circle, but releasing yourself from the struggle of it, This itself can be aplyied to the main proceess of Art. Creating a beautiful composition will not happen just with three quick lines, It's a pacient process where (As you draw) you must think before actually make another line. Don't move the pencil if you don't have at least an idea of where you want it next, and when you move it, it is tracing the path you want? if not, stop it! correct the angle and keep going but always without erasing what you did before, because that can be corrected in the future. That's something I have learned from the old masters. We want perfection, there's nothing wrong in that, but in order to archive it you must see perfection also in the most weirder figures, because world itself is weirder.
You already have the practice! now you have to add a little of consciousness into the equation and voila! you will have a perfect cicle! to resume, my best advices will be: Draw slow but with flow, don't finish the line if it is not going where you desire (Stop, visualize the circle and fix the angle of the line, and then continue the same line where you was, always without releasing the pen, we don't wan't to broke the line).
One last thing that actually happened to me. I realize time ago that my hand was to "Heavy", is not something people talk about because actually most people is "Light handed", so is hard to notice. Was kinda hard to maintain my hand pulse right because of this, so I actually have to kinda rise my elbow. Is uncomfortable at first, but it will come handy with practice, this way to draw helped me with my lines!.
These are my clircles, they are far from Perfect circle made with a tool, but I'm okay with that because they have more rytm and flow than a "Perfect one", wich, at the end, is what makes my work recognizable. So they are more useful for imprinting energy in the composition!