3:20 AM, Tuesday June 21st 2022
Indeed it is fine to ghost all the way back - and I encourage you to do it. The only distinction I would make is that when you ghost back, do not attempt to picture the vanishing point itself - rather, focus on the other lines of the same set (both those that have been drawn, and those that have not yet been drawn). Focus on how they converge, and how the line you're about to draw ought to be oriented in order to converge with them. This reliance on things that are physically close to the box allows us to infer useful information based on what is present in our drawings, rather than trying to keep track of what could be hundreds off vanishing points in a complex scene, when tackling actual illustrations.