Inquiring Minds Want To Know

If one has a cat door so the cat can go in and out at will, do all the neighborhood cats (squirrels, rats, raccoons) also come and go at will?

Now that we're letting George taste the great outdoors, we've started contemplating the cat door option, but I can just imagine the wild and crazy parties at 2 am that he would throw for the neighborhood critters.