## page was renamed from ImprovingPageLoadPerformance This page is a collection of ideas for improving the [[MeasuringPagePerformance|Time To Interact]] of pages on Launchpad.net. <> = Which Users Should We Target? = == Launchpad Commercial Users == Over the coming months we will be focusing on performance issues for commercial users who work for OEMs, ODMs, and our own field engineers. What follows is a rough outline of how these people interact with Launchpad: * Most of this user group is in Asia. Their network times are quite slow. * Many do not use Launchpad because their company has standardized on internal bug tracking software. * They overwhelmingly browse Launchpad with Windows XP and IE7. * When they do check Launchpad they do so as an anonymous user. * They hold independent accounts, not part of a private company account. * They use Launchpad bugs to find and occasionally file issues in Ubuntu core packages. * They use search extensively. How do these factors affect TTI? * Reducing network time is particularly important for OEM users. They suffer 10x the network latency of our European users. If Europe takes 150ms to load 1 kilobyte, Asia takes 1500ms. * IE7 has a limit of two connections per-domain for parallel downloading of resources. This makes the site much slower to load. How do we reduce TTI for these users? * IE-specific improvements to script loading and caching will help, such as the `deferred` script attribute. * Because commercial users use IE7, domain sharding and reducing the number of requests on a page become very important. * Proper caching is very important to reduce round-trips, because they are so expensive. * Reducing payloads will help. * Removing SSL would help. * Generation time may be important, depending on how many cache hits and/or misses they generate. = What Can We Improve? = == Network Time == Network time is roughly 65% of Time To Interact. === Server-side Improvements === * Make Librarian downloads '''reuse connections'''. Not doing so now. Reason unclear. * ''A big win on the [[http://www.webpagetest.org/result/100308_5SED/1/details/|site homepage]], the profile pages, and possibly on the project pages. --mars'' * ''Rough eyeballing the webpagetest.org results suggests this accounts for 20% of network time. --gary'' * Why is launchpad.js not cached ([[http://www.webpagetest.org/result/100308_5SEE/1/details/cached/|example]])? * ''Making it cached as expected seems as if it would speed the example page by 33% of the total TTI! That's IE-specific though. --gary'' * Remove requests for the same static file from both the application and root domain within the same page (e.g. fetching an image from `bugs.launchpad.net` ''and'' `launchpad.net` in the same page) * ''Easy to fix. --mars'' * Send real `Cache-Control` headers for `/@@/spinner` and friends (the headers are missing entirely). Without these the resource falls back to Etags to determine freshness, and the Etags are always changing, making the resource impossible to cache. * Tracked by the YSlow analyses. * ''An easy win. --mars'' * Remove SSL. * Private projects and teams would remain under HTTPS; everything else would be under HTTP. This would also probably suggest drastically reducing the amount of time that authentication cookies are kept, from a year to more like half an hour. This would reduce the risk of stealing session cookies in HTTP, and the openid server (HTTPS) could remain with long timeouts, perhaps, so that the user experience can still not require frequent logins. Switching back and forth from HTTP to HTTPS may be tricky, but we can probably find low-hanging fruit for initial implementations (e.g., force redirect to HTTPS if something private wants to be rendered over HTTP; offer redirect to HTTP if nothing private is rendered over HTTPS). * We might have three level of security: normal (HTTP), restricted (only available for users with specific permissions but over HTTP), and private (only available for users with specific permissions and over HTTPS). The restricted level (HTTP) balances greater security risk for faster connections. * ''Ballparking the webpagetest.org results seems to suggest that SSL cost is very significant--from 33% to as much as 75% of the network costs. Addressing this seems like it would be a big win. --gary'' * ''If most Launchpad visits are anonymous, say 75% of them, then we can consider dropping SSL just for anonymous access. That will benefit a massive portion of the Launchpad userbase, and is very easy to do from a technical standpoint. --mars'' * Domain sharding for librarian lookups (need numbers to back this up) * ''Try fixing librarian connection reuse first. --mars'' * Static assets use each individual asset's lastest bzr commit revno in their URL instead of using the latest `+icing` revno. * ''This would greatly increase cache reuse for occasional visitors, especially people who view the site only a few times each month. Would be nice to have numbers here, but awstats does not give us a returning visitors count. --mars'' * CDN or poor man's CDN to help US, Southwest Pacific, and South American users (What are options for poor man's CDN? Identify items we could and could not put on CDN; skeptical it would be a lot) * Uglification: Remove newlines, spaces, and comments from returned HTML. * ''Here is a [[http://perfectionkills.com/experimenting-with-html-minifier/|study]] of a complete HTML minifier. It only leads to an average 3-5% reduction in payload. Not worth the effort. --mars'' === Client-side and In-page Improvements === * Load JavaScript using the `defer` attribute * ''Sidnei says that there is no harm in doing this, and that some browsers other than IE do make use of it. --mars'' * ''That would be IE4+ (!) and Firefox 3.5+ (see http://groups.google.com/group/browserscope/browse_thread/thread/4c88af1b4ef17197?pli=1). But apparently Safari 4 and Opera 10 as well (http://hacks.mozilla.org/2009/06/defer/). Apparently the 'async' attribute is also an option, but it's unclear to me if it requires the HTML5 doctype (http://www.nczonline.net/blog/2010/08/10/what-is-a-non-blocking-script/), maybe just adding the attribute is enough --sidnei'' attributeSidnei says that there is no harm in doing this, and that some browsers other than IE do make use of it. --mars'' * Lazy-loading JavaScript for key pages * ''This does not actually improve Time To Interact, it just makes the page render sooner. The user still has to wait to interact with the page itself. Our own experiments with this feature have been an utter failure, but Yahoo manages to use it very very well on their homepage. --mars'' * Change inline script node ordering to maximize parallel downloads * ''Need a simple experiment here: save a page using the browser, put it in a public place, tweak node order, test with webpagetest. --mars'' * Cut payloads * Remove Mochikit * Determine benefit for IE7. Probably of minimal benefit for other browsers because of parallel download. * Examples like [[http://www.webpagetest.org/result/100308_5SEE/1/details/cached/|this one]] seem to indicate that it is even unimportant for IE. * Push all of the common /@@/ resources into our sprites file. * ''For example, we needlessly served 2.4GB of `/@@/edit` icons to 1.8 million page views in the month of March 2010. --mars'' * Enforce a baseline of five requests when loading any Launchpad page: 1. HTML (structure) 1. launchpad.js (behaviour) 1. combo.css (presentation) 1. sprites.png (common content) 1. spinner.gif (common content) == Generation Time == Generation time may account for '''35% of TTI for anonymous requests''', and '''40% of TTI for authorized requests'''. [[attachment:bzr-project-request-times.ods|(Here are the calculations for viewing http://launchpad.net/bzr.)]] Generation time is drastically reduced when you hit the squid caching server instead of the Zope application servers. Squid will typically returns results in less than 200ms, ''one tenth'' of the time it takes to get data from the application server in the example above. This effect is most visible to authenticated users, who always hit the application server. ''(What percentage of page requests are authenticated? --mars)'' ''(The percentage is probably less important than the fact that most of the user base we support are authenticated. The only anonymous thing that has much weight on it at all would be the pages that support the "anonymous download of release" usecase. --gary)'' * Chameleon (saves about 10-15% faster during generation, according to tests done by Gary Poster Q4 2009; specific results not available.) * ''This would mean roughly 5% (10-15% of about 35%) total TTI savings given the current scenario. Not bad, but not the first thing to attack. --gary'' * Memcached, MongoDB, or other similar technology * cache page elements, such as bug comments and tag clouds * cache ETags (can be used for launchpadlib) * session data * We could use zservertracelog analysis to identify pages viewed a lot, and pages that are responsible for the most time in the appserver. We could then use this to determine the pages we should look into optimizing. To do this, we should use pageid to identify and aggregate pages, rather than URLs. * XXX add info on Stuart's results * Detailed profiling numbers * Time spent in session authentication? * Time spent in the database? * Time spent rendering the response? == Render Time == Assuming that render time is everything spent outside Network and Generation time, Render time would account for a very small percentage of the Time To Interact in a typical Launchpad.net page: perhaps a few hundred milliseconds. The notable exceptions are the Bug``Index and Bug``TaskIndex pages, which load some page components using JavaScript after the DOM `load` event. * Load less-important page elements later (already started in 3.0 UI)