Technically a permit to carry a firearm, open or concealed, does not apply to this federal law (18, 921 and 18,922). Neither of those federal statutes mention any sort of permit. What 18,922 does mention is "if the individual possessing the firearm is licensed to do so by the State...", which means a license to possess a firearm, not one to carry a firearm. This just goes to show how poorly and confusing statutes can be written.
At any rate, you do not have to have a permit on your person when walking or driving by a school and carrying openly, or concealed in your vehicle. You just have to have the permit. As for this federal statute being enforced, this virtually never happens. State and local law enforcement is very reluctant to enforce this law. And in any case, it has already been tested in court, as I recall, and found to be unconstitutional (I'm pretty sure that is correct).
I wouldn't worry about walking or driving past a school while armed, especially in Virginia.