I allow my utility, insurance, and phone companies to take money for my bills. They always send me an email with a link to the bill, if I want to check. The advantage is that when I travel there's no worry about the bills getting paid. But then, I've always got enough in my checking account to cover it.
I could go to the bank's site instead, but for utility bills that means doing it manually each month since the amounts are always different. By trusting the utilities to do it, I save that bother. And I can always contest a charge with the bank if the utility screws up and won't admit it. I've never had that happen.
I pay my taxes on line, but for that I go to their web site and enter the amount and the date it is to be paid manually each time. I guess I trust the IRS less than I trust the utility companies. And for taxes, I need to move money into the checking account. That's more than I keep there on a regular basis.
Given your experience with the mortgage company, paying electronically directly from your bank, where you control the amount and the date, and have electronic proof of date paid, seems like the way to go.