The workaround is to migrate the server to another host that doesn't have the high latency issues.
The permanent solution:
To resolve this issue, we have to check the SFP, physical port and cable. Replacement of the SFP on the affected port which will improve signal quality and act as a good corrective measure to mitigate CRC errors beyond clearing the errors is required