[Experiment] Pref-Flip Experiment: Fission Process Count Fx 92.0 to 94.0 Nightly

User Story

Delivery Type: Pref Flip Experiment

    What is the preference we will be changing

dom.ipc.processCount.webIsolated

    What are the branches of the experiment and what values should
    each branch be set to?

- Treatment process-count-4 50%:

Value: 4

This treatment branch uses up to four content processes per site.
        
- Control process-count-1 50%:

Value: 1

The control branch uses one content process per site.
        

    What version and channel do you intend to ship to?

100% of Nightly Firefox 92.0 to 94.0

    Are there specific criteria for participants?

* Require telemetry environment's "fissionEnabled" == true so we only draw from the ~65% of Nightly users who are already using Fission.

NOTE: We want this experiment to run for all Fission Nightly users (so 50% of them are treatment, 50% are control). Therefore we need to target 100% of Nightly because the fissionEnabled filter will select just the Fission users.
Countries: all

Locales: all

    What is your intended go live date and how long will the experiment run?

Aug 03, 2021 - Aug 31, 2021 (28 days)

    What is the main effect you are looking for and what data will you use to
    make these decisions?

What is the main effect you are looking for and what data will you use to make these decisions? What metrics are you using to measure success?

Same as all of our Fission Beta experiments, the most recent being https://experimenter.services.mozilla.com/experiments/fission-beta-91-experiment/

We want to be able to monitor the difference in stability and performance metrics between clients with and without Fission enabled. The Fission Experiment Plan (https://docs.google.com/document/d/1ZXAHep4v1lW-ZpLMtnTXAtbXSNP8Q-gr-RjUha2Yb14/edit#heading=h.asctkf7cbxa7) has a detailed list of telemetry probes (and expectations for each probe: up, down, or flat).

We plan to monitor these metrics:

Memory metrics:

- `MEMORY_TOTAL`
- `MEMORY_UNIQUE_CONTENT_STARTUP`
- `CYCLE_COLLECTOR_MAX_PAUSE`
- `CYCLE_COLLECTOR_MAX_PAUSE_CONTENT`
- `GC_MAX_PAUSE_MS_2`
- `GC_MAX_PAUSE_MS_2_CONTENT`
- `GC_MS`
- `GC_MS_CONTENT`
- `GC_SLICE_DURING_IDLE`
- `GC_SLICE_DURING_IDLE_CONTENT`

Performance metrics:

- `CHECKERBOARDING_SEVERITY`
- `CHILD_PROCESS_LAUNCH_MS`
- `CONTENT_FRAME_TIME_VSYNC`
- `FX_NEW_WINDOW_MS`
- `FX_TAB_SWITCH_COMPOSITE_E10S_MS`
- `KEYPRESS_PRESENT_LATENCY_MS`
- `INPUT_EVENT_RESPONSE_MS`
- `TIME_TO_FIRST_CONTENTFUL_PAINT_MS`
- `TIME_TO_FIRST_INTERACTION_MS`
- `PERF_PAGE_LOAD_TIME_MS`

Engagement metrics:

- Number of users submitting pings
- `browser.engagement.total_uri_count`
- `browser.engagement.tab_open_event_count`
- `browser.engagement.max_concurrent_tab_count`
- `SUBSESSION_LENGTH`
- `CONTENT_PROCESS_COUNT`
- `CONTENT_PROCESS_MAX`
- `LOADED_TAB_COUNT`

Stability metrics:

- Main process crash rate (per 1000 usage hours), 32-bit and 64-bit
- Content process crashes (per 1000 usage hours) (excluding ShutDownKills), 32-bit and 64-bit
- `crash_tab_ui_presented`
- `crash_subframe_ui_presented`
- `oom_crashes` (per 1000 usage hours), 32-bit and 64-bit
- `shutdown_crashes` (per 1000 usage hours)

Do you plan on surveying users at the end of the delivery? No.

 Estimated Total Enrolled Clients: 360,000 (assuming we enroll 15% of 1.2M Beta + Dev Edition users running Windows or macOS)

    Who is the owner of the data analysis for this experiment?

[email protected]

    Will this experiment require uplift?

False

    QA Status of your code:

TBD? This experiment is not much different from our Fission Beta experiments like https://experimenter.services.mozilla.com/experiments/fission-beta-91-experiment/

    Link to more information about this experiment:

https://experimenter.services.mozilla.com/experiments/fission-process-count/