MDC leader, Morgan Tsvangirai says opposition parties have to unite to oust Mugabe.
Mujuru, the widow of Zimbabwe's first post-independence army general, was fired from Mugabe's government and the ruling ZANU-PF party in December 2014.
Before being sacked, Mujuru had been seen as Mugabe's favoured choice of successor. She launched her own party in March.
"We want the people of Zimbabwe to work as one not to be divided. Today we found a way of working together to deal with issues affecting us," Mujuru told the rally.
Thousands of Tsvangirai's Movement for Democratic Change supporters and Mujuru's newly-formed Zimbabwe People First chanted "Mugabe must go" at the rally.
Others hoisted placards reading "Where are the promised jobs?" "Stop Abductions" and "No to bond Notes" -- referring to the pending introduction of controversial bond notes, a local token currency equivalent to the US dollar.
Zimbabwe has been hit by a series of street protests triggered by an economic crisis that has left banks short of cash and the government struggling to pay its workers.
Also on Saturday, state media reported that there would be delays – once again – in paying the salaries of civil servants.
The Herald newspaper reported that August salaries for army personnel, police, teachers and health workers would be delayed by at least a week. In July, government employees staged a strike over late pay.
"It is a fact that we have revenue challenges," Prisca Mupfumira, Zimbabwe's public service minister, told the newspaper when announcing the new pay delay.
In June only police and security forces were paid, while other civil servants received a $100 advance with the news their salaries had been delayed.
Zimbabwe spends at least 80 percent of its revenue on state workers' wages, according to officials, while about 90 percent of the population has no formal employment.