Did some more research and found a thread on the Airstream forum. Guy sounded like he knew a lot about the subject so I will quote him below. All in all, Ive concluded that the lower voltage (12.8) Im seeing when hooking in the trailer batteries to the TV is normal. Also, I could reduce some of the voltage drop by cleaning or replacing my connectors, which I will do. I have no further expectations that the TV will swiftly recharge a discharged trailer battery due to the resistance in the long run of 10GA wire. Best I can expect is a slow recharge by the TV over time. Realistically I can expect the TV to keep my batteries topped off, with the typical long 10GA charge line, and several points of added connector resistance, including the ground path back to the TV.
FWIW the Airstream thread:
tow vehicle charge line to trailer battery.. - Airstream Forums
What happens is that it just takes longer to bring the RV battery up to full charge.
The vehicle charging system senses the chassis battery and often has a separate lead for this. Once that battery is up to a proper float voltage, the system keeps it there.
So the situation is that the charging source quickly gets the chassis battery back to a float voltage yet the RV battery is still discharged because current to it has a lot of resistance to get through. The RV battery will then continue to charge because it is below the charging system's float voltage. The charge current will be limited by the resistance in the feed line plus the fact that the source voltage is not very high. That is most of the reason why it can take quite a while on the road to get the RV battery charged up.
When the battery gets up to a full charge, it will no longer accept current at the float voltage level and the very low currents nearing that point mean very low loss in the charging line. So it gets there, it just takes a while.
Note that the ground return on the RV battery is often a good source of resistance itself and a common source of problems.
Based on the above theory, on my next long trip I think I will test the TV and trailer batteries over numerous hours of towing and see how the voltage changes, if at all.