Going to a Q-Jet would really help - and set the floats in it a little lower than normal too - about 3/16" lower.
But if you stick with the Holley - If it's not a center-fed type, convert it. As suggested modify the external bowl vents, set the floats about 1/8" lower than the bottom of the inspection hole.
Go to a good speed shop - get the "white whistles" for inside the bowl, they extend the vent inside to the middle of the bowl, get "jet extensions" to get the jets to suck from the middle of the bowl, and get spring loaded needles and seats. And get the tapered floats designed for road courses.
Holleys are great street and strip carbs - hard to beat them, but off road's a different story. They don't like to be sloshed or jounced or up-hilled, and they have such tiny passages that a dirt spec you wouldn't notice in your eye will plug them up.
Get the bowl thumb screws, extra bowl gaskets, and a gas cup, you'll be taking them off to clear the grit out every once in a while on the trail.
The biggest advantage of the Q-Jet is the float pivots are in the center of the carb - makes a huge difference.
They used to call the Q-Jet "Quadra-Flush" because they have lots of parts, and "mechanics" didn't understand them and were afraid of them. Times have changed.
Get the book on Q-Jets by HP Books - very helpful and informative.