Ryan Murphy says the 200m backstroke final at the Tokyo Games was "probably not clean" after he lost his Olympic title to Russian Evgeny Rylov.
However, the American swimmer quickly backpedalled when asked if he was accusing any of his rivals of cheating.
Murphy won gold in the 100m and 200m backstroke at Rio 2016 but Rylov won both titles in Tokyo, setting an Olympic record in the latter on Friday.
"I've got 15 thoughts, 13 of them would get me into a lot of trouble," said 200m silver medallist Murphy when asked by a reporter shortly after his event if he had any concerns about his races in relation to doping.
"It is what it is. I try not to get caught up in that. It is a huge mental drain on me to go throughout the year, that I am swimming in a race that's probably not clean and that is what it is."
At a later news conference, however, Murphy said he was only talking about doping in swimming in general and congratulated Rylov and British bronze medal winner Luke Greenback.
"I need to be clear...my intention is not to make any allegations here,"he said.
"Congratulations to Luke and Evgeny. They did an incredible job, they're both very talented swimmers.
"At the end of the day ... I do believe it (doping) is still big in swimming and it is what it is."
Rylov was surprised by Murphy's initial comments and at his own media conference said he always supported clean sport.
"I always do the doping tests ... I would not be able to forgive myself if I had taken something," he said.
"I don't know how to react to this. I haven't been accused of anything."
A Russian team spokesperson declined to comment on Murphy's remarks.
USA Swimming had no immediate comment.
Russian athletes are competing at this year's Olympics as the Russian Olympic Committee as part of sanctions for doping scandals.
They do not have their flag raised or their national anthem played when they win events in Tokyo.
Swiss swimmer Jeremy Desplanches, who won bronze in the men's 200m medley, said he had no concerns about his race.
"Of course there are some strange cases. But when someone is better than you, it is so much easier to just say he has doped," he said.
"But sometimes people are just better than yourself."
Australian Associated Press