[Performance] Opening search for the first time on a big account is causing the app to hang massively

[hannojg]

If you haven’t already, check out our contributing guidelines for onboarding and email [email protected] to request to join our Slack channel!

---

What performance issue do we need to solve?

In a customers meeting the customer was trying to open the search for the first time in NewDot. It caused the app to hang up, and it was a very bad user experience (see the recording):

Screen.Recording.2024-07-31.at.16.25.28.mov

What is the impact of this on end-users?

User perception of the app is very bad, as it's super laggy, basically not usable (the browser even warns that the app is hanging).

List any benchmarks that show the severity of the issue

The customer shared a profile trace with us:

Firefox 2024-07-25 10.42 profile.json.gz

(note: the trace also contains other test cases as well)

The customer had ~15k reports loaded in onyx when these tests were conducted, although in focus mode only 6 chats were displayed.

Proposed solution (if any)

None yet, I will go through the profile and see what can be optimised, what exactly caused those lags.

List any benchmarks after implementing the changes to show impacts of the proposed solution (if any)

not available yet

Platforms:

Which of our officially supported platforms is this issue occurring on?

• Android: Native
• Android: mWeb Chrome
• iOS: Native
• iOS: mWeb Safari
• MacOS: Chrome / Safari
• MacOS: Desktop

Version Number: v9.0.11-5
Reproducible in staging?: not tested
Reproducible in production?: yes
Email or phone of affected tester (no customers): customer
Logs: See performance file
Notes/Photos/Videos: See attached video
Expensify/Expensify Issue URL: n/a
Issue reported by: @hannojg
Slack conversation: https://expensify.slack.com/archives/C05LX9D6E07/p1721919928992729

View all open jobs on Upwork

Issue Owner

Current Issue Owner: @sakluger

[melvin-bot]

Auto-assigning issues to engineers is no longer supported. If you think this issue should receive engineering attention, please raise it in #whatsnext.

[hannojg]

cc @sakluger (feel free to assign me as I (or someone from my team) will work on this ticket!)

[marcaaron]

Curious what is happening with this especially to see if it is related to large numbers of reports (seems likely as there are 15k reports). Gonna assign myself in case @hannojg needs any assistance.

[sakluger]

No updates

[sakluger]

No update

[hannojg]

A large part of this hanging is due to us building the search options the moment we open the search page:

• App/src/pages/ChatFinderPage/index.tsx

  Lines 78 to 91 in 4bac100

  	const searchOptions = useMemo(() => {
  	if (!areOptionsInitialized || !isScreenTransitionEnd) {
  	return {
  	recentReports: [],
  	personalDetails: [],
  	userToInvite: null,
  	currentUserOption: null,
  	categoryOptions: [],
  	tagOptions: [],
  	taxRatesOptions: [],
  	headerMessage: '',
  	};
  	}
  	const optionList = OptionsListUtils.getSearchOptions(options, '', betas ?? []);

I am thinking about how to solve this here, by moving the operations in a first step to a separate thread/web worker:

• https://expensify.slack.com/archives/C05LX9D6E07/p1723454316012939?thread_ts=1723209631.007999&cid=C05LX9D6E07

[hannojg]

I just opened a PR hat shows a PoC for using web workers for running the search, which will unblock the main thread:

• [WIP] WebWorker Demo (to improve search performance) #47432

I am now seeking approval on this new technical design here.

(Note: I am OOO for the rest of the week and will continue next week Monday 19.08)

[sakluger]

Moving to weekly for now since Hanno is out until Monday.

[hannojg]

Can you please also assign @chrispader? he will work on that as well (Chris please comment 🫶 )

[chrispader]

Commenting for assignment :)

[sakluger]

Thanks for helping, @chrispader. I'll be OOO for the next week. Hopefully we'll have some more details from the investigation when I return!

[hannojg]

We were able to reproduce the bigger lags when running on a virtual machine, where the hardware is more constraint than on our recent MacBook Pro models directly. (note: we tested against a recent adhoc build that contained some search improvements).

The thing that takes the most time when opening the search page / chat finder page is:

• calling createOptionsList
• calling getSearchOptions (on the results of createOptionsList)
• calling triggerUnreadUpdate also took ~1-2 seconds

We will come up with solutions for improving the performance of these functions (in the current single threaded architecture).
(Some of these solutions might blend with the solutions we are working on here)

[sakluger]

Hey @hannojg - fyi, if you don't want an issue to auto-close, just remove the # before the issue URL on the linked PR. We have automation set up to auto-close GH issues when you link it in the PR using this format:

[sakluger]

I'm also removing the payment hold from the issue title since it's not relevant here.

[melvin-bot]

Reviewing label has been removed, please complete the "BugZero Checklist".

[melvin-bot]

The solution for this issue has been 🚀 deployed to production 🚀 in version 9.0.88-7 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:

• perf: reduce loops in getValidOptions / separate to getValidReports #55298

If no regressions arise, payment will be issued on 2025-01-30. 🎊

For reference, here are some details about the assignees on this issue:

• @hannojg does not require payment (Contractor)
• @chrispader does not require payment (Contractor)
• @hannojg does not require payment (Contractor)

[hannojg]

Greta, thanks for the insight. Working on a new exciting improvement here which should not only help with search, but also with the sidebar calculation (so app start time):

• perf: Calculate shouldReportBeInoptionList once with a bitwise flag #55742

[melvin-bot]

The solution for this issue has been 🚀 deployed to production 🚀 in version 9.0.89-8 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:

• perf: optionsToExclude array to loginsToExclude map #55348

If no regressions arise, payment will be issued on 2025-02-04. 🎊

For reference, here are some details about the assignees on this issue:

• @hannojg does not require payment (Contractor)
• @chrispader does not require payment (Contractor)
• @hannojg does not require payment (Contractor)