“I carry a gun so I don't have to fight.” This excuse, or variations thereof, have been around for as long as I can remember.
Real confidence stems from capability, and predators know what that looks like, and they usually avoid it. They're looking for an easy meal, not a fight. Learning to give and take a punch (or arm bar) gives you that confidence.
Not all fights are gun fights. Just because you have a gun on you doesn't mean you should pull it out, let alone use it.
Even when a fight does involve guns, you may have to “earn your right to draw”; that is, control the physical space in order to gain the time/distance needed to do so.
Within a certain distance, in some situations, it makes a lot more sense to go hands on with your attacker than it does to try to beat them to the draw. Remember, the bad guy gets to initiate the conflict. You can't go around drawing down on everyone that acts suspiciously (unless you're security guard in Brazil?), so beating them to the draw isn't always an option.
A good H2H school will give you a workout and get you accustomed to stress. A good school will put you into positions that are very uncomfortable, not just physically, but also psychologically. Learning to work through it will help keep you calm in a real fight, and is a real confidence booster.
The good news is that you don't have to be a black belt in secret ancient arts to survive an attack. Of course, even if you train every day you don't have a guarantee. But it does swing the odds back towards you if you're fit enough that you won't have a heart attack, some training so that actions are instinctual and your awareness level is high so that you can be proactive.
What sort of training have you taken and found to be worthwhile?