d.Backup() is 100% read IO !

[rockit-ba]

Question.

When I use d.Backup() to backup, read IO will be 100% immediately. It is almost inevitable. When I frequently insert and delete key-value pairs, I backup every 100,000 or so times.

It is very likely to happen when I use d.Backup() for the second or third time. My machine is 4 cores and 8 GB. At that time, the CPU was very low, but the read IO was 100%. I had to restart the server because it could not do anything.

code：

func (d *badgerPersist) RaftBackup(sink raft.SnapshotSink) error {
	var number uint64
	if _, err := os.Stat(filename); os.IsNotExist(err) {
		number = 0
	} else {
		data, err := os.ReadFile(filename)
		if err != nil {
			return err
		}
		number, err = strconv.ParseUint(string(data), 10, 64)
		if err != nil {
			return err
		}
	}

	log.Info("start to backup data", "lastVersion", number)
	newLastVersion, err := d.Backup(bufio.NewWriter(sink), number)
	if err != nil {
		return err
	}
	log.Info("finish to backup data", "lastVersion", newLastVersion)
	err = os.WriteFile(filename, []byte(strconv.FormatUint(newLastVersion, 10)), 0644)
	if err != nil {
		return err
	}
	return nil
}

[rockit-ba]

this is my config

func initOps() badger.Options {
	return badger.DefaultOptions(storagePath).
		WithNumGoroutines(runtime.NumCPU() * 2)
}

and i do GC

func dBGc(db *badger.DB) {
	for range gcTicker.C {
	again:
		err := db.RunValueLogGC(0.5)
		if err == nil {
			log.Info("badger gc success")
			goto again
		}
	}
}