The office of federal prosecutor Harald Range said the NSA had failed to provide enough evidence to justify legal action.
The allegations of NSA phone-tapping came out in the secrets leaked by US whistleblower Edward Snowden about large-scale US surveillance in 2013.
German-US ties were severely strained.
When the allegations were made the White House gave no outright denial, but said Mrs Merkel`s phone was not being bugged currently and would not be in future.
Mrs Merkel told the US government angrily that "spying between friends just isn`t on".
On 4 June last year Mr Range said "sufficient factual evidence exists that unknown members of the US intelligence services spied on the mobile phone of Chancellor Angela Merkel".
But in December he revealed that the investigation was not going well and he had not obtained enough evidence to succeed in court.
A statement from Mr Range`s office on Friday said "the accusation cannot be proven in a legally sound way under criminal law".
It said "the vague statements by US officials about possible surveillance of the chancellor`s mobile telecommunication by a US intelligence service - `not any more` - are not enough to describe what happened``.
A German parliamentary committee is also investigating NSA surveillance, but it has not managed to get much help from US officials either, Germany`s Spiegel news website reports.
Mr Snowden worked as a contractor for the NSA but fled to Hong Kong from the US in May 2013 after revealing extensive, global internet and phone surveillance by US intelligence.
After speaking to The Guardian newspaper in Hong Kong he flew to Moscow, where he is still living.
The US charged him with theft of government property, unauthorised communication of national defence information and wilful communication of classified communications intelligence.
More about:
