Like Mark said, the rules do not define anything to be out of bounds. The rules only tell you what you do if a disc is out of bounds. It is up to the tournament director to decide what areas will be out of bounds. For situations like that a TD needs to make a string line, or just declare that the water is casual. There is no way to define where the water begins and the muck ends for something like that.
The rules do say you give the benefit of the doubt to the thrower, so if it was understood that the water was o.b., I'd give the thrower the benefit of the doubt and say the muck is in bounds.