Analyse Time to First Byte issue with Magento ecommerce site running on a dedicated server

I am looking for a magento and server expert who can offer advice on how to improve the 'Time to First Byte' on our magento e-commerce site We are running a fast specification, dedicated server with a 8 core Intel 2680 processor, Raid 10 SSDs, 32 GB Ram. I am getting a TTFB of around 2.5 to 3.5 seconds depending upon the page, so I know the basic hardware is likely not to be the issue. My Magento developers have implemented the usual server config items have been implemented like mysql caching, memcache, APC etc I would like an independent person who is an expert in these matters to undertake some analyse on the server and to make recommendations which I discuss with my Magento developers on changes which will increase the speed of the TTFB. The primary focus on this project is the TTFB issue so I'm not particularly looking for Varnish or front end caching solutions. Depending on the findings and the quality of the analysis, I would also be interested in discussing the costs of implementing the changes you recommended.