Is this occurring AT the supply spigot where you connect your hose?
Does the spigot have a 'vacuum breaker' installed?
If YES..sounds like vacuum breaker is faulty.
They will spit water out but should only do so when water supply is turned off
If this is NOT what you are asking about......?????
What is system pressure?
Side Note: closing down on faucet does NOT reduce pressure....just restricts FLOW