After Navajas was arrested by the police earlier in the day, Communications Minister Isabel Fernandez broke the news to the media.
"The president has decided to remove Marcelo Navajas as Health Minister to avoid any interference in the work of the justice (department) and impeding the investigations," said Fernandez.
Current Deputy Health Minister Eydi Roca will "temporarily fill the position," she added.
Other officials under investigation for taking part in the scheme have also been fired, said Fernandez.
Two officials were arrested on Monday, including the head of the Health Ministry's legal department, Fernando Valenzuela, and the executive general director of the AISEM, the agency in charge of procuring medical equipment, Giovani Pacheco.
Two advisers from the Inter-American Development Bank (IDB), who approved the purchase, were also arrested on Wednesday.
Justice Minister Alvaro Coimbra pledged there will be no impunity.
"It doesn't matter whether (the accused) is minister, deputy minister or director, they are going to be tried," he said.
The scandal emerged after Bolivian officials purchased 170 emergency respirators from the Spanish firm GPA Innova, using IDB funds.
Press reports on the steep premium paid for the ventilators, bought at 27,683 U.S. dollars each, sparked social media backlash against the South American country's government, which received IDB funds to purchase the devices in May.
"I will seek jail and order the full weight of the law against those who have taken a single cent. Every penny of corruption must be returned to the state," caretaker President Jeanine Anez had written on Twitter late on Tuesday.
In televised comments, she said that she had ordered a thorough investigation "no matter whom it brings down."
The IDB said in a statement on Wednesday it was concerned about "possible irregularities in the purchase of respirators" made by Bolivia's Health Ministry using financing from the bank.
The lender said earlier in May it had financed the country's purchase of the ventilators as part of 82 million U.S. dollars in funding for Bolivia to fight against the spread of the coronavirus.
(CGTN, with input from Xinhua, Reuters)