For us the potential fuel mileage is insignificant compared to what we need to be accommodating for our traveling style and what we like.
I would love to pull a Mini or a Beetle but we really utilize the extra storage provided by the 1/2 ton P/U with a cap. Although on shorter trips it is not necessary usually it is.
I can see us with a smaller toad in the future but it will be in addition to the P/U not a replacement for it.