The continued use of Pontins Brean Sands to house hundreds of Hinkley Point C workers is reportedly having a "big detrimental impact" on the local economy, according to one tourism firm.