Haptic Feedback is a short vibration in response to an interaction with the device. It is a way that the phone can provide a physical response to something being pressed.
To enable or disable Haptic Feedback:
Settings > Preferences > General > Toggle ON / OFF