When your opponent attempts to move pieces onto your side of the board you must challenge those pieces. If you allow your opponent to move unchecked onto your side of the board you are asking for trouble. If you are trying to execute a certain opening or gambit, it may still be wise to challenge these trespassers that have come to your side of the board.
You need to defend against these pieces that come to your side of the board either by attacking them, trading, or finding away to drive them away and make them retreat. Any of these methods are good in defending yourself against these trespassers. Letting your opponent place pieces on your side of the board and taking no action allows your opponent to gain an advantage that he or she will likely hold throughout the entire game. If you ignore these pieces that your opponent has placed on your side of the board it will allow your opponent to bring even more primary pieces over to your side of the board and will limit your possible movements.
It is especially important to attack these pieces or drive them away when they are placed in the center of the board. Controlling the center of the board is one of the basic principles in the game of chess. If you see your opponent placing unchecked pieces there you must challenge or attack if you want to have a chance to win the game. Often times if you are able to force your opponent to retreat these trespassing pieces you will gain a tactical advantage. You will have the momentum, or tempo, on your side while he or she is in the process of retreating.