Privacy issue for single woman staying in a PG (Paying Guest)

Hi, I am a single woman staying for a short term (3months) in a co-living PG space in Bangalore. It is a studio room with two bedrooms, and the other bedroom is currently empty. When I moved in 2 weeks ago I was informed that if someone moves in, it will be a female. Today while I was working from home, two men came to my door, knocked and made a big ruckus, and then used their spare keys to open the lock. They told me that they were showing the room to a potential customer - a man. There was no woman with them. Fortunately I had latched the door as well, but now I am afraid for my security and privacy. When I leave the room they can easily enter in. I am feeling extremely unsafe. The house rules do not state that they can enter at any time for this purpose, and I am worried for my safety. When I go to office I have no choice but to leave my belongings here, and if they have the keys then it is not safe for me. Not to mention, as a single woman it is definitely not safe for me to allow strangers (men) to come in. The door has only a doorknob with a keyhole, there is no way for me to add extra safety measures (like a padlock for instance). My question is - are they legally allowed to enter the room whenever they want? Is there anything I can do for my own safety?