The struggle against coding bugs

I hear that when a bug gets created, one needs to find its origin. I read that one should only change one thing at a time rather than scrapping a body of code taking the bug with it. Nobody ever makes a case for it, just accepting it as "the one true way". I theorize that being prone to bugs is due to flawed overall systemic design, so the presence of a bug should cause you to seek improvement in the design, the location of the bug telling you where the change in the design should be made, rather than going on a bug hunt.