The judge found there had been widespread irregularities in the 21 May vote.
This is the first election to be legally challenged since Malawi’s independence in 1964. A new vote will take place within 151 days.
There have been regular anti-government protests since the election.
The court ruled that Mr Mutharika will stay in power until a new election takes place.
Why was a legal challenge filed?
President Peter Mutharika won a second term in May with 38.6% of the vote.
But opposition candidates Lazarus Chakwera, who came second, and Saulos Chilima, who finished third, went to court to argue that the election was not fair.
They said the way the election was conducted was full of irregularities.
The judges, who arrived in court in the capital, Lilongwe, under military escort, upheld the complaint on Monday.
According to the official results, Mr Chakwera, who claims he won the election, garnered 35.4% of the vote as the flag bearer for the Malawi Congress Party.
Mr Chilima, a former ally of the president, got 20.2%. He was the candidate for the UTM.
There is a high level of anxiety in the country following months of tense clashes between the police and opposition supporters.
Many schools were closed and some public transport had been suspended ahead of Monday’s court decision.