The gap between simulation and hardware is called the reality gap. A policy that exploits a perfectly rigid virtual floor or a frictionless joint model falls over on the first real step.
The standard remedy is domain randomisation: during training, randomly vary masses, friction, motor strength, control delays, lighting and sensor noise. A policy forced to cope with thousands of slightly different worlds tends to handle the real one, which is just another variation.
Other tools are better actuator models identified from measurements on real motors, and fine-tuning on a small amount of hardware data. Sim-to-real is the reason a quadruped can be trained on simulated gravel in hours and then walk on actual gravel.
