Speed of DNN site has always been an issue however you can minimize the delays till some extent and this is true that a DNN site can run as fast as any other site built with PHP.
There are two major factors:
Dead sites
Yes, your site dies or sleeps if its not opened by a user for a certain time(in minutes of most of the servers). The IIS shuts down the process and cleans the memory in that case resulting a very long time on next first hit.
Rendering Time
The loading time of a page depends on the rendering time, design level things like CSS, image optimization, Cache settings, compression settings etc. Also, disabling the features that you don't need in a site will reduce the burden.
1. Skin should be fully CSS based
2. Use some other menu provider (Native navigations i.e; Nav, Solpart etc are slower)
3. It also depends what modules you're using in your site. Text/HTML is the fastest module.
4. For every new dnn project, we always optimize it according to Mitchel's most useful post in his blog: http://www.mitchelsellers.com/blogs/articletype/articleview/articleid/283/how-i-get-my-dotnetnuke-sites-to-run-so-fast.aspx
5. Also, there's a very good document about this topic found here DotNetNuke Performance Configuration Best Practices by Mitchel Sellers
6. Never let your application sleep.
I hope this helps!