Waters was photographed wearing a long, black jacket with a crimson wristband while performing at Berlin's Mercedes-Benz Arena on May 17.
He also pointed a toy machine pistol towards the audience. Nazi uniforms, flags, and emblems are not permitted to be displayed in Germany.
Its rules, however, allow for exclusions for artistic or educational purposes.
"We are looking into allegations of inciting public hatred because," said police spokesman Martin Halweg.
The costumes used on stage could be used to glorify or defend Nazi power, upsetting the community's tranquility."
Waters wore a crimson wristband with two black crossed hammers on a white circle at previous concerts dating back several years.
The markings are similar to those seen on clothing in the 1982 film The Wall, which was based on the novel of the same name.
Pink Floyd's album of the same name, which featured activist and fellow musician Bob Geldof.