Pingbacks and trackbacks are basic WordPress elements that notify you as and when your web page or blog receives a link. They may come across as efficient tools, but there are other tools like Google Webmaster which help in checking the links of your website.
Enabling them would put an unwanted load on your server because every time someone would try to link up to your website, it would generate requests from WordPress back and forth. These features are extensively misused while targeting a website with DDoS attacks.
Thus, disabling them will help you speed up your website.