Is this a new HEI swap or did it work fine and then just start happening?
I'm thinking it should be on a 20 amp fused key on circuit or possibly a inline fusible link. Factory it appears there's no fuse in the ignition circuit but a fusible link. I don't believe most factory HEI setups had relays. GM didn't really start using relays till they went to fuel injection.
Are you attempting to get rid of the computer controlled carb/ ignition disaster and upgrade? If so read on, if it was working fine and just "happened" then Jim Lou's advice is probably it. Here's a wiring diagram, there is a resistor inline from the alternator to the ignition module, it "should" be yellow with a brown wire going to the alternator with an inline resistor (the alternator/ battery light might be the resistor). | Repair Guides | Wiring Diagrams | Wiring Diagrams | AutoZone.com
Do you have an HEI distributor with the coil on the cap? It should only have 2 wires going to the cap, a 12 volt key on hot and a tach signal. You will have 3 wires going from the coil to the distributor. You want to focus on the wires from the engine wiring harness to the coil, not the coil to the distributor. If you have an HEI system that doesn't have a coil on the cap, what's the donor application so I can find the right wiring diagram. Did you check and see if the bat wire is hot all the time even with the key off? I'm wondering if you maybe wired it to an always hot terminal on the starter relay or solenoid.
These are "generaic" HEI swap distributor instructions but they should help http://www.4wd.com/Instructions/PD/DUI_Install.pdf
I looked at a wiring diagram and it made my head hurt but here's where I would start looking. There's a green and a yellow wire coming out of the ignition switch, originally the yellow went to the ignition module, coil, and t-ed into a brown wire that goes to the alternator. Unplug the distributor/ ignition coil (to eliminate a distributor problem), at the distributor check to see that when the key is off it's cold, turn the key to the crank position and also the key on positions, it should be hot. If that's the case hook that wire up to the bat terminal and see if your problem is solved.
If you still have the computer/ most of the wiring for the emissions disaster you can really simplify it with HEI and a MC 2100 and loose a ton of wire. You should only need a bat wire to the distributor that's hot in the start and run positions, Tach signal wire from the tach terminal on the distributor/ coil to the tach, oil and temp sender signal wires from the sensors to the gauges, electric choke wire that I believe should be hot in the run position, a wire to the starter relay and then to the solenoid that's hot in the start position, and a wire to the alternator that's t-ed into the distributor bat wire with a resistor (on a GM the resistor is an indicator light, the wires going in/ out of the light are hot in the run position but the light grounds through the alternator and only grounds when the alternator isn't working, I don't know if that's how the jeeps are set up of if there's an actual inline resistor) . Hopefully the auto zone wiring diagram will help that make sense.
I'm by no means an expert but that should start you in the right direction. The pre 4.0 carbed YJ's have a mess of wiring under the hood that can be scary till you realize how much of it you really don't need.