You didn't say which model of Brake Buddy you have, but yes a slight delay is normal and does not affect overall braking. The system first has to sense when the coach is braking and then apply the toad brakes, same thing in reverse when you release the coach brakes.
2009 Damon Tuscany 4072 Cummins ISC 360
2007 Jeep Wrangler 4dr / Blue Ox towing system / Brake Buddy Advantage Select
Full-timer and loving it every day!