"Gen Qamar Javed Bajwa, you packed up our government, which was working well, and put the nation and the country at the altar of your wishes", Mr Sharif told the gathering - the largest since the 2018 election.
In his address, the former prime minister also accused the chief of Inter Services Intelligence, Pakistan's premier intelligence agency, of being involved in conspiring against his government.
Nine major opposition parties formed a joint platform called the Pakistan Democratic Movement (PDM) last month to begin a nationwide agitation against the government.
Railways Minister Sheikh Rashid Ahmad has cautioned that Pakistan Muslim League-N supremo Nawaz Sharif could meet the fate of Muttahida Qaumi Movement (MQM) founder-leader Altaf Hussain due to his criticism of the Pakistan military leadership.
The Punjab Assembly speaker advised Nawaz Sharif to focus on his medical treatment, for which he fled the country. He blames generals and judges for what he says were trumped up charges.
Sharif called for Khan's resignation and ending the military's interference in politics.
PPP chairman Bilawal Bhutto Zardari also took the stage to lash out at PM Imran Khan and his government, saying that the cricketer-turned-politician did not have any solution to the problems of the poor. "Your time is up".
"We have come out for the supremacy of the law", said Maryam Nawaz, the daughter and political heir of former Prime Minister Nawaz Sharif, speaking from atop an SUV that was showered with rose petals as she left for Gujwanwala from nearby Lahore.
Federal Minister for Railways Sheikh Rasheed opened a new Pandora's box on Saturday when he claimed that former prime minister Nawaz Sharif met someone secretly at a hotel in Turkey.
Commenting on the PDM rally, the minister said he had stopped judging political rallies on the basis of numbers.
Mr Khan, who came to power on an anti-graft platform, said on Friday that he was not afraid of the opposition's campaign. The prime minister claimed the rally was aimed at blackmailing him to drop corruption cases against their leaders.