Changing Employers but staying with same client

[freza0]

My situation

My employer A->My prime vendor->End client (current situation)

I flied multiple h1b with diff employers and got approved H1b from Employer B, not Employer A.

I have to change my contract to

Employer B->My prime vendor 10.0.0.0.1 192.168.1.254->End client(same client)

I have a non-compete with Employer A that I should not work with the client for 18 months after contract.

If I terminate the contract with Employer A and join Employer B(with same client) will it back bite me any time in future?