Regardless of what your answer is, they will continue traveling in the same direction as before. If you talk to the manhunters, they will ask if you have seen any bandits recently.
It is possible to increase relation with all manhunters by helping them kill bandits, providing they are about to lose, though relation with manhunters seems to have no impact on anything. The color of their units is quite similar to, but a little lighter than, the color of bandits. Manhunters aren't pledged to any kingdom, and therefore don't fight in any wars except against bandits. They will never upgrade themselves as Manhunters belong to the 'manhunter' faction, while Slave Drivers and the remainder of the troop tree all belong to the 'slaver' faction, requiring the player's control to enable them to switch factions. They can also be upgraded and rank up, becoming stronger and gaining sturdier armor, better weapons, and improved horses. Since they can't be recruited from anywhere, Manhunters can only be hired by freeing them from captivity. They usually are victorious against Looters, but are almost always defeated by Sea Raiders.
They are generally average fighters with blunt weapons that they use to knock out bandits, and they wear relatively light armor while riding horses. Manhunters are law enforcers who hunt the Looters, Deserters and other Bandits that roam throughout Calradia.