= Use forking for the LP Serve processes =
I propose to change how '''bzr lp-serve''' is spawned. Instead of starting a
new process from scratch for every connection, we would instead have a running
process that would be asked to fork for the new connection.
'''As a ''' developer<
>
'''I want ''' to handshake with bzr+ssh quickly<
>
'''so that ''' I can get bzr content as fast as possible<
>
== Rationale ==
Mostly motivated by graphs like this:
https://lpstats.canonical.com/graphs/CodehostingPerformance/
Which shows that while it takes ~0.6s to do the ssh handshake, there is still
another 3+s to actually start talking to the bzr smart server. Manual testing
of 'time bzr lp-serve --inet' takes as much as 2s on an unloaded machine.
This should benefit most people who connect to Launchpad via the ssh interface.
== Stakeholders ==
* Any user of Bazaar and Launchpad
== Constraints and Requirements ==
=== Must ===
* It must preserve the existing security and isolation for the codehosting processes.
* It must also be faster than the existing system, or it hasn't actually improved anything.
=== Nice to have ===
=== Must not ===
* Violate the security constraints (allow user A access as user B, etc.)
== Success ==
We will be done when we can see that the average connection time to Crowberry
drops significantly. The existing connection time graph should be sufficient
for that.
* https://lpstats.canonical.com/graphs/CodehostingPerformance/
== Thoughts? ==
There are some options for a pre-fork manager (keep at least N unused processes
running, when a connection comes in use one and start another). The main reason
I'm thinking a long-running process that forks itself is so that we actually
avoid the startup time, which helps total load on the system, rather than just
improving interactivity.
My current implementation thought is that the {{{execCommand}}} functionality
would talk to the running process (starting it if necessary), and request it to
fork. Which would then give back a new PID and the path-on-disk for
stdin/stdout/stderr of the new process. This would then be abstracted into a
standard twisted IProcessTransport (though stdin, etc would actually be a FIFO
and not the raw file-handle from a child process). This is instead of the
current {{{spawnProcess}}} call.
The Conch server would know how to contact the running process via shared
configuration. The server can hang on a named pipe (in config) and take
commands like 'spawn USER' or 'shutdown' and get back the new PID and paths for
the client.
--- wooo -- great, RobertCollins