It's bad because it starts attracting people who are only into the "traceur" look about parkour and not parkour. This leads to people calling things parkour that aren't parkour (i.e. climbing buildings, being reckless). The attitude that those people bring to parkour isn't something anyone should want.
At the risk of derailing this topic, I feel like I should point out that climbing, whether trees, rocks or structures, is one of the central types of parkour movements. The other basic movements being running, jumping, and vaulting (and swimming, which also fits the "A to B efficiently, human body w/o equipment" definition, though the overwhelming majority don't consider swimming parkour). They are all methods of getting from one point to another that can be used with either efficiency or expression as one's primary focus. All of them can be practiced safely, or not. Climbing buildings carries the additional risk of alarming or distressing the public and thus potentially damaging the reputation of parkour. It is, however, an essential skill that should be practiced in places where the safety and public exposure risks are minimal, IMO.