There are different reasons for that problem! You might have duplicated the bricks but placed them to close to each other (or better, they overlap each other). What happens now is that the object bounds of each brick overlap and thus collide and one brick will bounce away from the other. What helps is carefully analysing whether there are bricks overlapping or not. Also it could be that you forgot to move some of the duplicated bricks, so there are multiple bricks at same location. The same happens then. I hope this will solve your problem,

Kind regards.