Voltage will tell you state of charge of your batteries. As with any measure, it needs to be done properly and interpreted properly.
There are two ways to use a DVM such as pictured. One is to watch how it changes when the water pump or furnace comes on. How much and how fast it drops and recovers will tell you about the state of health of the battery.
For state of charge, check the voltage after no significant charging or discharging for at least a half hour. Start thinking about charging the batteries when voltage gets down to 12.4v, try not to let it get below 12.2v if you can, and put charging as a high priority when it gets down to 12.0v.
See What’s the battery doing? Start with a DVM
. The Equus 3721 linked there has one more digit than that shown above which is nice if you like numbers. Another gadget I like is described at Nifty gadgets: “The car’s electron pressure, weatherglass, show”
. It has voltage only to the tenth but the clock and temperature is useful as well.
As noted, you do need to make sure you are measuring the voltage of the proper battery and that you have good connections. These devices draw a few tenths of a watt so their power needs usually won't cause wiring voltage drops. I have seen some interesting voltage drops through rusty fuse box connections and also due to dissimilar metals on battery binding posts. For the most part, though, plugging one of these devices into a handy cigarette lighter outlet and reading the voltage when there are little or no battery loads works well.
Also note that some folks really like numbers and get one of those $200 or so integrating ammeters that try to estimate state of charge by comparing discharge energy to recharge energy. These are fun but they need to be programmed properly and you need to be aware of their limitations. With a $15 voltmeter and a bit of common sense (and, perhaps, a bit of experience), you can do just as well at monitoring your battery.