The storm has now moved off the north-eastern coast of Cuba towards Florida, where warnings are in place.
South Carolina is to start evacuating more than a million people.
Matthew, now a category three hurricane, is predicted to hit the US east coast later in the week.
The hurricane also hit Cuba, but early reports suggested it is impact was not as hard as in Haiti, where there were winds of 230km/h (145mph), heavy rain and dangerous storm surges.
At least 10,000 people were in shelters and there were reports of overcrowded hospitals suffering shortages of fresh water, Mourad Wahba, the UN special representative for Haiti, said.
The storm knocked down communications and blocked roads, hampering emergency efforts.
The collapse of a bridge cut off the only link between the capital, Port-au-Prince, and the southern part of the country, and officials admitted it would be difficult to reach the region.
Laura Sewel, an aid worker with Care Haiti, said: “The mobile network has also gone down. So what this means for us is that we’re out of contact with our staff right now which is quite difficult and the bridge going down means it will be harder to move materials.”
In the port town of Les Cayes, the situation was “catastrophic”, with streets flooded and many houses without roofs, deputy mayor Marie Claudette Regis Delerme said. She herself had to flee a meeting when a gust ripped off the building’s roof.
Images showed people walking in shoulder-high water, with relief workers saying that other coastal communities were also under water, including Les Anglais.
Fonie Pierre, director of Catholic Relief Services for Les Cayes, told AP: “Many people are now asking for help, but it’s too late because there is no way to go evacuate them.”
Heifer International, a non-profit organisation, said farmland and businesses caught in Matthew’s path had been devastated.
Haiti is one of the world’s poorest countries, with many residents living in flimsy housing in flood-prone areas.
The UN said the country was facing the “largest humanitarian event” since a huge earthquake in 2010.
More than four million children could be exposed to hurricane damage, Unicef said, warning of the spread of waterborne disease.
Interim President Jocelerme Privert said earlier that some people at sea or who had not “respected alerts” had died, but he gave no more details.
In the neighbouring Dominican Republic, at least four people were killed by collapsing walls and mudslides.
A hurricane alert was in place for five provinces of Cuba, with residents moved from low-lying areas. There were no immediate reports of serious damage.
Matthew is likely to remain a powerful hurricane until at least Thursday night as it sweeps through the Bahamas towards Florida and the Atlantic coast of the southern US, the National Hurricane Center said.
South-eastern Florida was under hurricane and tropical storm warnings, and South Carolina Governor Nikki Haley declared a state of emergency.
Hurricane Matthew is the region’s most powerful since Felix in 2007.