How to prevent Memory Percentage increase over time?

[gotbletu]

hey i notice that if i leave viddy running for a few hours, the memory % would increase. Is there a way to prevent this?
viddy starts out at 0.1% after a few hours it would be over 4.0%.
the watch command is always at 0.0% even after a few hours.

Not sure if is due to the viddy time machine feature that records all the changes but if so, maybe add an option to disable it to prevent memory % increase.

# example
viddy -t man ffmpeg
viddy -t "ps -o pid,user,%mem,command ax | sort -b -k3 -r | grep -i viddy"

---

viddy 0.3.4-1
System: Host: arcolinux Kernel: 5.10.83-1-lts x86_64 bits: 64 Desktop: Xfce 4.16.0 Distro: ArcoLinux

[kjk11]

Same problem here, great application, but the memory usage slowly keeps going up until the system runs out of RAM.

[onedr0p]

It's probably worth setting like a 100 history limit by default but maybe make it configurable @sachaos ?

[sachaos]

I'll try to solve this issue in this weekend.

I could not try. I'll do this in this week.

[paulie-g]

I see that this feature branch was never merged into master. I would respectfully suggest that you do so, and cut a release. A lot of people use watch/viddy for long-running monitoring. The time machine functionality is neat, but not at the expense of unbounded memory consumption.

[dylan-tock]

I see that this feature branch was never merged into master. I would respectfully suggest that you do so, and cut a release. A lot of people use watch/viddy for long-running monitoring. The time machine functionality is neat, but not at the expense of unbounded memory consumption.

Just had a box get into a state that required a total restart because of this.

[onedr0p]

Any update @sachaos ?

[sachaos]

@onedr0p Hi thank you for ping.

I think there are many solutions.

1. limit snapshot count.
2. store history in file.
   • SQLite ? Store data in SQLite (so that process memory usage does not increase) #104
3. Remove 0 diff snapshot.
   • I've merged history: skip snapshots with no changes (+0 -0) #99

[fedorkanin]

I use viddy for git log monitoring on MacOS, this is the command I use:

viddy --interval 1s --no-title --skip-empty-diffs 'GIT_PAGER=cat git log --all --oneline --graph --decorate --color'

The issue with unbounded and increasing memory consumption (after a day or so it can be more than several gigabytes) in my case is not solved with --skip-empty-diffs. I do not really understand what is taking up so much memory and would be happy to cooperate by providing logs if requested.

As for my workflow, an ideal solution would be to limit the snapshot count, either via a flag or a global configuration.

[sachaos]

I've implemented feature to save and load history in viddy v1.0.0-rc.2.
We have been storing results in memory until now, but since we switched to using SQLite, we expect a significant reduction in memory usage. If you encounter any bugs or issues, please let us know.

#117 (comment)

[paulie-g]

I've implemented feature to save and load history in viddy v1.0.0-rc.2. We have been storing results in memory until now, but since we switched to using SQLite, we expect a significant reduction in memory usage. If you encounter any bugs or issues, please let us know.

I can't speak for absolutely everyone, but the people in this thread who knew what they wanted asked for an ability to put an upper bound on in-memory storage. This is because their data is ephemeral and not worth hammering flash, ie they are using viddy as a drop-in replacement for, say, watch. Saving data to SQLite is - at best - orthogonal and, unless you're using an in-memory table, potentially even more unsuitable to their use case.

[sachaos]

@paulie-g Oh, you're right. I apologize for the misunderstanding.
Yes, the SQLite backend won't resolve this issue.
We'll need to limit the history length.

Thank you!