It is most likely efflorescence. Efflorescence is nothing more than salt that has wicked its way up through the pavement and solidified on the surface of your brick. While it is unsightly, it is not physically harmful to your paver in any way.