Dart Analyzer very slow/stuck #55281
Comments
lrhn
added
the
area-analyzer
keertip
added
type-performance
|
@DanTup , do you see this? Have there been more reports of this kind? |
|
/ cc @bwilkerson |
|
I've not seen any other reports like this. 100% CPU certainly does not look healthy if it's persisting for a long period after initial analysis. @JonasJW a few questions:
|
|
We were having the same issue and using the Removing it completely seems to have solved the issue for us, are you using it too, and can you try to remove it to confirm it comes from there if you are ? |
|
/cc @scheglov |
keertip
added
P2
|
@DanTup Thanks for the response and sorry for the late reply. code --status output:To be honest, I'm not quite sure what you mean by your question about symlinks or how I can find out. Yes, this issue seems to have started just recently without making any noticeable changes such as updating SDKs, installing extensions, etc. Here are further screenshots, I hope this is the server timing page you meant.
If you have any recommendations what I can try to fix this issue, I'm happy to try that. As mentioned, I have already tried uninstalling Flutter/Dart, VSCode, etc. at the moment I don't know what else I could try. |
|
For us updating |
|
@JonasJW thanks - the page I meant was the one marked "Timing" on the left. Can you also confirm whether you're using any analyzer plugins? A few comments above suggest You should be able to check for symlinks by running |
|
@DanTup thanks for the clarification! I don't use When I run Is that anything out of the ordinary? Here the screenshots of the "Timing" page:
|
That one doesn't look like a problem, but are there any that point back up the tree (eg. creating cycles or including other large parts of the disk in the the path)? It may be useful to enable the analyzer instrumentation log (this file can get very large - be sure to turn it off afterwards) and reproduce the issue, and see if there is anything in the log file that looks out of place while this happens (for example exceptions, or paths you would not expect to be analyzed with this project open being analyzed). You mentioned this only happens on large projects - are any of them public projects that I could test with (or that sharing logs from would not include anything sensitive)? |
|
Hi @DanTup, I apologize for taking so long to respond. I still have this issue and it's really slowing down my development process.
Here are a few symlinks, some point up but it doesn't appear to me that they would create a cycle
In total there are over 550 symlinks. Does any of this sound problematic? I also tried the analyzer instrumentation logs but I can't really find anything that appears out of place. However, this document seems to log all kinds of stuff and I'm not sure how to analyze it or what to look for. Yes, unfortunately, this project is private and can't be shared. I could probably share the analyzer instrumentation logs privately with you, if that would help. I recently tested this project on a windows laptop and the issue occurs as well. (Thus, I think completely reinstalling my mac won't even help fix it). These issues don't appear on new Flutter projects. Maybe I could try to download a big public project and see if I have problems with other bigger projects as well? If not it must be something specific with my project but I'm clueless about where the issue could be. |
Nothing above looks like an issue to me - the issue would be if the link points further up the tree so that walking the tree could cause endless cycles.
The most useful thing would be knowing what's being logged during the periods where performance is bad. So if the bad performance is during startup, it would be the start of the logs (until the first
I can't accept anything confidential (instrumentation logs can contain contents of opened files and paths/errors for other files in the workspace), but if you're able to reproduce this on a copy of the project with anything confidential removed (and perhaps only a single test file open), you might be able to get a log that doesn't contain anything confidential you can share. Something I forgot to ask earlier - can you confirm whether you have this option ticked in the VS Code (User or Workspace) settings?
|
|
Understood, I will go back to the logs and look for these progress events. If I have something to share I will follow up on it here. The |
FWIW, my recommendation would be to not use that setting (and I should update the text). Although it sounds better for large projects, it was really added for a fairly specific case (command line editors that would provide the current working directory - which may be the user home dir - as the workspace folder). While the server will start up faster for very large workspaces, opening and closing files from different projects will trigger re-creating the analysis roots which can trigger "initial" analysis for those projects. It's beneficial if you're opening a folder that contains 100 projects and maybe only opening files from a handful, but if you're jumping between a large portion of the projects in the workspace, it's probably worse overall. |
|
@JonasJW what is your current experience with analysis performance? Are you still experiencing this weird behavior? |
|
@mraleph yes, I'm still experiencing performance issues |
|
@JonasJW are you able to reproduce this issue with any public projects? It might be easier to narrow down with an instrumentation log, but that log will contain parts of source code from the open projects. If it only occurs with this one specific internal project, is it possible to make a copy of it and see if you can narrow down what causes it (for example if it's made up of many packages, can you remove some of them to narrow down if it's caused by a specific package/set of packages)? Do you also know if this issue occurs when typing in any file, or just some subset? This issue could be similar to #56307 which may be caused by the number of files that need to be reanalyzed as you modify a file (see #56307 (comment) and #56307 (comment)). |
|
Maybe a related issue |
|
We have a pretty large project (internal, 200k lines), and this issue is pretty hard on productivity. The worst of it comes from typing new class properties. The analyzer will get stuck on these for up to 20-30 seconds. Here's my 
(The top consumer seems to be the Android Emulator, which is probably fine) Here's the Dart process (if relevant): Anything else I can do to help debug this issue? |
|
@realth000 thanks for the file - there are a lot of slow @knopp thanks, I can reproduce fairly long analysis when making a single-keystroke change to that file too (not to the point of needing to restart the server, but my testing was fairly limited). I've opened a specific sub-issue for this with my timings and a CPU profile at #60293. |
Also add benchmark where lots of hover requests are fired, mimicking IntelliJ behavior when holding ctrl and moving the mouse over imports. I'm getting these results on my machine: ``` 4 files / CodeType.ImportCycle: Initial analysis: 1.249438 Completion after change: 10.143719 4 files / CodeType.ImportChain: Initial analysis: 1.367604 Completion after change: 9.936909 4 files / CodeType.ImportExportCycle: Initial analysis: 1.233644 Completion after change: 10.011695 4 files / CodeType.ImportExportChain: Initial analysis: 1.226382 Completion after change: 9.875991 4 files / CodeType.ImportCycleExportChain: Initial analysis: 1.262932 Completion after change: 9.995607 ``` notice the low number of files (in the chain*) or if it even is a chain etc doesn't appear to change anything for the time it takes to become responsive again. Notice that in #55281 (comment) I noticed that something similar happened for `edit.getFixes`, but I'm yet to actually reproduce that. Change-Id: Iff04124825c6ea2f759b4d048a0f2988709eaebf Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/413981 Commit-Queue: Jens Johansen <[email protected]> Reviewed-by: Phil Quitslund <[email protected]>
TL;DR: By only responding (for real) to the newest hover request a benchmark that used to take ~10 seconds now takes ~0.6 seconds. Details: In #55281 (comment) I noticed that some of the performance issues people are seeing fom the analyzer server stem from many requests being fired, each taking longer the wait before the next is issued, naturally building up a longer and loger queue before responses are made. Their it was for `edit.getFixes`, but I'm yet to actually reproduce that. I did reproduce for hover requests though, so here I'm debounsing those hover requests. I'm hooking into the system that already debounces completion requests and following the pattern used there. On my machine, running ``` out/ReleaseX64/dart-sdk/bin/dart \ pkg/analysis_server/tool/benchmark_tools/big_chain_benchmark/legacy_many_hover_requests.dart ``` I get these numbers: Before CL (taken from the parent CL that introduces the benchmark): ``` 4 files / CodeType.ImportCycle: Initial analysis: 1.249438 Completion after change: 10.143719 4 files / CodeType.ImportChain: Initial analysis: 1.367604 Completion after change: 9.936909 4 files / CodeType.ImportExportCycle: Initial analysis: 1.233644 Completion after change: 10.011695 4 files / CodeType.ImportExportChain: Initial analysis: 1.226382 Completion after change: 9.875991 4 files / CodeType.ImportCycleExportChain: Initial analysis: 1.262932 Completion after change: 9.995607 ``` With CL: ``` 4 files / CodeType.ImportCycle: Initial analysis: 1.322501 Completion after change: 0.569768 4 files / CodeType.ImportChain: Initial analysis: 1.347588 Completion after change: 0.635748 4 files / CodeType.ImportExportCycle: Initial analysis: 1.492362 Completion after change: 0.575233 4 files / CodeType.ImportExportChain: Initial analysis: 1.331619 Completion after change: 0.576199 4 files / CodeType.ImportCycleExportChain: Initial analysis: 1.342130 Completion after change: 0.608064 ``` Change-Id: Ide3dbc4dfb885af9e1c46a763338ba545c0ab7db Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/413982 Reviewed-by: Phil Quitslund <[email protected]> Commit-Queue: Jens Johansen <[email protected]>
johnniwinther
added
the
model-performance
|
@realth000 thank you for the data! |
|
@jensjoha I restarted the dart analyzer server after I have some git submodules in my repo, those are also dart projects. You can clone my project here https://github.com/realth000/tsdm_client if you'd like to. Some new info I found yesterday: the performance got much better when I disabled most configs in
Before, for the first and second config I chose "all files types" and |
|
FWIW I have this issue too. I'm running Dart/Flutter/Serverpod in VS Code in Windows 11 |
|
@parnesen please check the instructions in #55281 (comment) to collect a diagnostic report after you've reproduced the issue you're seeing. If you can reproduce on something open source / public, then please provide details and repro steps. |
|
I have a suspicion that this is related to barrel files. This issue describes the problem very well: #50369. I started experiencing this problem more and more as my project grew, and now I have around 300 files in my Also, I think this comment explains the issue that might be the root cause for me personally in my project: #50369 (comment) |
|
we don't have barrel files in our source code and it has the same behavior, however we have some gigantic generated files, even though they're not analyzed, I'm still thinking that it could be the source of the problem (the generated files are from the auto_route_generator and injectable_generator) |
|
Do we have a time-line for this issue? I want to upgrade my Flutter to 3.29, but this issue makes 3.29 unusable. |
^ I do not think version 3.29 is related to this issue - we are seeing this constantly for But yea, I think this should be the very top priority for the sdk team. 2 minute build_runners + analyzer getting stuck every other minute makes Dart feel like a severely handicapped language atm. At least for mid-large scale flutter development. |
2 minute |
|
This issue is conflating too many things together. First of all, I think we have two separate problems on our hands. One problem is that some people indicate a regression in Flutter 3.29 / Dart 3.7 release. To the best of my knowledge we still don't have any clear idea of what could have caused a regression, what this regression is and what can be done to address it. I suggest that any reports and discussions about Flutter 3.29 regression go into a separate issue: #60335 The second problem is long analysis times. There are might be different reasons for it - but one of the common pattern that we see is: editing code in a library which is transitively depended upon by large amounts of code. When you edit such library you cause reanalysis of all libraries which depend on it. Some files are naturally like this1, but there are there are several other ways to get your code base into such state. Common case is barrel files, where you create a file which imports large amount of libraries and then re-exports them all. However barrel files are just a special case of this problem. Routing is another case (I suspect that's what @melWiss hits): one of the common ways to structure routing means you end up creating a library which contains definitions of your routes - and this library both imports (directly or transitively) most of the libraries that define UI of your app and at the same time is imported into large number of libraries which define the UI. This means whenever you edit one library which is imported by the library which defines routes - you cause invalidation and reanalysis of all libraries which in turn import the library which defines route. The problem with such dependency graphs is that the coarse invalidation strategy employed by the analyser does not scale for them. @scheglov is working on addressing this, but we don't have fully concrete timeline for fine grained invalidation landing in the analyzer. So the best you can do today is to try and identify if your project falls into this trap and try to restructure dependencies between your files in a way that avoids it (e.g. move away from barrel files, rewrite your routing in a way that avoids cyclic dependency between god file defining routes and libraries which define UI). @jensjoha have written a tool which tries to analyze the code base and extract strongly connected components from it. You can try running it on your code base to get insights. Example of running it: $ dart pub global activate -sgit https://github.com/jensjoha/dependency_insight/
$ dart pub global run dependency_insight:dependency_graph $PWD/libFootnotes
|
|
@jensjoha can you make the project I don't use barrel files a lot so I think it is not a barrel files issue 
but I have part of for blocs 
And I have freezed, json serialization and auto route |
By "not analyzed" do you mean that they are "excluded" from analysis in the analysis options file? |
|
I have all generated files excluded but the same issue Note: I have auto fix 😁 , auto save 😳 and auto format 😂 in my vscode |
|
Are these |
In retrospect the name of the "exclude" entry was unfortunate, at best, as it appears to be misleading. I suspect that many users expect that it means that the file will never be opened and analyzed, but as Konstantin pointe out that isn't the case. "Excluded" files are only unread if they aren't reachable from any analyzed files. |
Yes that's what I meant @bwilkerson |
|
Question: Does it benefits performance if the project is separated into different packages and organize them using For example, each feature/module is a separate dart package, i have seen this approach in other languages: packages/
- feature1
- pubspec.yaml
- lib/ ...
- feature2
- pubspec.yaml
- lib/ ...
- feature3
- pubspec.yaml
- lib/ ...
pubspec.yaml -> pubspec for entire workspace |
If the alternative is to have all of features/modules be separate packages but not part of a workspace, then yes. The benefit of a workspace is that there is a single version solve for all of the packages in the workspace, which means that it's safe for the analyzer to build the element model for each third-party package once and share them across all of the packages. Without that guarantee of a single version solve the analyzer has to build the element model for each third-party package once for every package that has a dependency on it. If the alternative is to have a single large package with all of the code in each of the separate packages, then no, splitting it into separate packages in a workspace won't change the performance of the analyzer. On the other hand, if splitting the code into multiple packages would help to reduce the number of circular dependencies between pieces of the code, then the reduction in circularities would improve the performance of the analyzer. |
|
We've been experiencing this problem for a while now. We are currently on |
It effectively analyses everything in the cycle as if it were in a single library. |
|
I have this problem since upgrading to Flutter |
and others
Dart & Flutter has suddenly become unusable slow in VSCode. Intellisense won't load within 30+ seconds, syntax highlighting won't update, jumping to code definitions loads infinitely, etc.
This seems to be especially the case on bigger projects, on a newly created project it works fine. I tried switching to Android Studio which seems to behave similarly (maybe a little better).
I tried uninstalling all extensions, even completely removing VSCode with any data stored and reinstalling it. As soon as I install the Dart extension, it is unusable. I also tried setting the following settings:
I'm on macOS 14.4, VSCode Version 1.87.2 (Universal),
In the Activity Monitor, there is the
dart:analysis_server.dart.snapshotprocess running with around 1 GB of used memory. I'm not sure if this is normal.Here is a screenshot from the Dart Analyzer (which I'm also having trouble opening)
dart infooutput:General info
Project info
Process info
Any recommendations on how I could possible fix this would be very welcome!
The text was updated successfully, but these errors were encountered: