The newspaper also says a Chinese company has requested a top minister to hand over a 5,000 acre area, including the site of the VoA station, reports say.
The US has chosen the location due to its regional strategic location for air-wave transmission experimentation.
There have been allegations the VoA facility had been used to gather classified military information in the region, and also that it had a secret runway.
It is being completely run by the US governments and is out-of-bounds for Sri Lankans.
Initially, it was spread in a 920-acre area tightly guarded using diplomatic immunity.
The proposal to construct the broadcasting station was accepted by the Sri Lankan government in 1983 and construction began in 1993.
Residents of the area have been staging protests, but a revised facility was constructed at the site, with its extent brought down to 520 acres.
When contacted, minister Malik Samarawickrama confirmed that the US was withdrawing the facility, but said he was unaware of a Chinese request for the land.
Any Chinese move to take over the land will be strongly opposed by India, which has been against the VoA station since its inception.