The bus, which was stopped on a busy road, was buried under rubble in Gwangju, some 270 kilometers (168 miles) southwest of the capital, Seoul.
Eight other people were injured, the ministry added.
The 16 demolishers on the site are safe and sound and did not suffer any injuries during the collapse of the building.
About 190 firefighters have been deployed for search and rescue, and police have joined in the rescue efforts, the ministry said.
The reason for the collapse was unclear, Reuters reported, citing officials at Gwangju Fire and Security Headquarters. The Ministry of Land, Infrastructure and Transport will investigate the cause of the collapse.
South Korea struggles with a poor infrastructure security record: a department store collapse killed more than 500 people in 1995 and a bridge collapse in 1994 killed 49 people. In recent years, the country has attempted to improve its security record.
This story has been updated to reflect the number of injuries.