A 318 car is possibly near the bottom of the list, in terms of value. Walk away? It depends what you're looking for. If the plan is to build a restomod, discarding the existing engine and trans, what difference does it make if it's numbers matching or not? If the car has been in a major front-end accident in its 50 year lifespan, it seems possible the radiator support was replaced.
It comes down to dollars. If the car is the desired starting place for a restomod and the price seems right, then go for it. All the little trim pieces and what not take a good amount of time to find and replace. If the car is complete, and doesn't have a great deal of rust to deal with, its value goes up.
If this were a factory 440 car, then I agree, it's value goes down without the stamped number, but not by a great deal. I don't believe a 318 car is worth a great deal effort to fake, so I don't think there is anything nefarious to worry about.