Rescuers managed to save at least 50 passengers and were still searching for more survivors, regional police official Bidhan Tripura told Reuters.
Low-lying Bangladesh, with extensive inland waterways and slack safety standards, suffers regular ferry disasters, with deaths sometimes running into the hundreds.
The bodies retrieved so far from the water include a baby, Tripura said. Police have seized the trawler and arrested two of its crew, he said.
Another police team, assisted by firemen and the Bangladesh Inland Water Transport Authority, is trying to salvage the ferry, he said.
Divers were also trying to retrieve any bodies trapped inside the vessel, he said.
The ferry was heading to Paturia from Daulatdia in Rajbari, located on the opposite bank of the river and 136 km (85 miles) northwest of Dhaka.