An Instagram live stream by a Kazakh blogger showed a fire blazing in the mayor's office in the city of Almaty, with gunshots audible nearby. Videos posted online also showed the nearby prosecutor's office burning.
Protesters appeared to have broken through security forces' cordons even though the latter deployed stun grenades whose explosions could be heard throughout the city centre.
Kazakhstan is a tightly controlled former Soviet republic which cultivates an image of political stability, helping it attract hundreds of billions of dollars of foreign investment in its oil and metals industries.
President Kassym-Jomart Tokayev accepted the government's resignation on Wednesday, a day after police used tear gas and stun grenades to drive hundreds of protesters out of the main square in Almaty.
On Wednesday a Reuters correspondent saw thousands of protesters pressing ahead towards Almaty city centre, some of them on a large truck, after security forces failed to disperse them with tear gas and flashbang grenades.
Atameken, Kazakhstan's business lobby group, said its members were reporting cases of attacks on banks, stores and restaurants.
The city health department said 190 people had sought medical help, including 137 police. City authorities urged residents to stay home.
The interior ministry said that government buildings were also attacked in the southern cities of Shymkent and Taraz overnight, with 95 police wounded in clashes. Police have detained more than 200 people.
A video posted online showed police using a water cannon and stun grenades against protesters in front of the mayor's office in Aktobe, the capital of another western province
The protests began after the government lifted price controls on liquefied petroleum gas at the start of the year. Many Kazakhs have converted their cars to run on LPG because of its low cost.
The government said the regulated price was causing losses for producers and needed to be liberalised. The president said it had botched the move.
Speaking to acting cabinet members, Tokayev ordered them and provincial governors to reinstate price controls on LPG, and broaden them to gasoline, diesel and other "socially important" consumer goods.
He also ordered the government to develop a personal bankruptcy law and consider freezing utility prices and subsidising rent payments for poor families.
He said the situation was improving in protest-hit cities and towns, including Almaty and the surrounding province, where the authorities declared a state of emergency.
In addition to replacing the prime minister, Tokayev also appointed a new first deputy head of the National Security Committee who replaced Samat Abish, a nephew of powerful ex-president Nursultan Nazarbayev.
Nazarbayev, 81, a Soviet-era Communist Party boss, ranKazakhstan for almost 30 years before resigning abruptly in 2019 and backing Tokayev as successor. Nazarbayev retains sweeping powers as the chairman of the security council; he has not convened the council or commented on this week's violence.
The protests began in the oil-producing western province of Mangistau on Sunday, after LPG prices more than doubled following the lifting of caps.
A source familiar with the situation said some workers at Mangistaumunaigas, a Kazakh-Chinese oil-producing joint venture based in the Mangistau province, were on strike, although this was not affecting output so far.
Tokayev declared the emergency in Almaty and Mangistau and has said that domestic and foreign provocateurs were behind the violence.
Almaty mayor Bakytzhan Sagintayev said the situation in the city was under control and security forces were detaining "provocateurs and extremists".
Kazakhstan's dollar-denominated sovereign bonds suffered sharp falls with the 2045 issue falling around 3 cents in the dollar and many dropping to levels last seen in 2020, Tradeweb data showed.
Like many emerging and developing economies, Kazakhstan has grappled with rising price pressures in recent years. Inflation was closing in on 9% year-on-year late last year - its highest level in more than five years - forcing the central bank to raise interest rates to 9.75%.
Some analysts said the protests - the most serious in the country in at least a decade - pointed to more deep rooted issues.
"I think there is an underlying undercurrent of frustrations in Kazakhstan over the lack of democracy," said Tim Ash, emerging market strategist at BlueBay Asset Management.
"Young, internet savvy Kazakhs, especially in Almaty, likely want similar freedoms as Ukrainians, Georgians, Moldovans, Kyrgyz, and Armenians, who have also vented their frustrations over the years with authoritarian regimes."