There's a lot of little things you can do to improve the page load speed of your website...really nit picky kind of stuff that seems to only change the slightest things...so does it matter? Well, yes! A lot of little issues can pile up into a big issue, and search engines like Google take note of these things, both for organic searches and paid advertising through Google Adwords.
So what to do? Well take a look at this helpful article that will show you how page load speeds effect your website rankings and what you can do fix them.
http://searchengineland.com/site-speed-ppc-performance-cant-ignore-slow-site-anymore-213029