A government official told the BBC that some of the more than 700 buildings shut down have already reopened after they were approved by inspectors.
According to a proposed new law, all preachers must have theological training before opening a church.
Pentecostal churches, often run by charismatic preachers claiming to be able to perform miracles, have grown rapidly in many parts of Africa in recent years.
Some are massive, attracting thousands of worshippers each Sunday, but others consist of tiny structures built without planning permission.
Church leaders have at times been criticised for using loud public address systems to attract worshippers.
Government official Justus Kangwagye told the BBC's Focus on Africa programme that they simply required the churches to meet "modest standards".
Some church premises exposed worshippers to unnecessary risks and could "cause danger to those worshipping," Mr Kangwagye told the Rwandan New Times newspaper.
The reported crackdown is being carried out by the local authorities with the support of the Rwanda Governance Board, according to the New Times.
Some 700 churches were thought to be operating illegally, the Rwanda Governance Board said in a recent statement.
The vast majority of Rwandans are Christians but many also follow traditional practices.