You don't say how much distance there is from the main breaker panel that is fed from the utility company to the pedestal you installed. If your house is where the utility wire is attached then the distance is from the house to the garage to the pedistal. Using #4 aluminum wire is equivalent to the current carrying capacity of #6 copper. It sounds to me like you have a voltage drop problem.
2007 Newmar DSDP 4023
Discovery is seeing what everyone else has seen and thinking what no one else has thought.
If you want to see what man made go East; if you want to see what God made go West.