• Immutable Page
• Info
• Attachments
• More Actions: Raw Text Print View Render as Docbook Delete Cache ------------------------ Check Spelling Like Pages Local Site Map ------------------------ Rename Page Delete Page ------------------------ Subscribe User ------------------------ Remove Spam Revert to this revision Package Pages Sync Pages ------------------------ Load Save SlideShow

Use forking for the LP Serve processes

I propose to change how bzr lp-serve is spawned. Instead of starting a new process from scratch for every connection, we would instead have a running process that would be asked to fork for the new connection.

As a developer
I want to handshake with bzr+ssh quickly
so that I can get bzr content as fast as possible

Rationale

Mostly motivated by graphs like this: https://lpstats.canonical.com/graphs/CodehostingPerformance/

Which shows that while it takes ~0.6s to do the ssh handshake, there is still another 3+s to actually start talking to the bzr smart server. Manual testing of 'time bzr lp-serve --inet' takes as much as 2s on an unloaded machine.

This should benefit most people who connect to Launchpad via the ssh interface.

Stakeholders

• Any user of Bazaar and Launchpad

Constraints and Requirements

Must

• It must preserve the existing security and isolation for the codehosting processes.
• It must also be faster than the existing system, or it hasn't actually improved anything.

Nice to have

Must not

• Violate the security constraints (allow user A access as user B, etc.)

Success

We will be done when we can see that the average connection time to Crowberry drops significantly. The existing connection time graph should be sufficient for that.

• https://lpstats.canonical.com/graphs/CodehostingPerformance/

Thoughts?

There are some options for a pre-fork manager (keep at least N unused processes running, when a connection comes in use one and start another). The main reason I'm thinking a long-running process that forks itself is so that we actually avoid the startup time, which helps total load on the system, rather than just improving interactivity.

My current implementation thought is that the execCommand functionality would talk to the running process (starting it if necessary), and request it to fork. Which would then give back a new PID and the path-on-disk for stdin/stdout/stderr of the new process. This would then be abstracted into a standard twisted IProcessTransport (though stdin, etc would actually be a FIFO and not the raw file-handle from a child process). This is instead of the current spawnProcess call.

The Conch server would know how to contact the running process via shared configuration. The server can hang on a named pipe (in config) and take commands like 'spawn USER' or 'shutdown' and get back the new PID and paths for the client.

--- wooo -- great, RobertCollins