ValueObservation and tracking regions

[Artem-Viveritsa]

I have the following function that tracks changes in the database for each object type:

var blocks = Dictionary<BlockType, [AnyBlock]>()

private func observe() {
    
    cancellable.removeAll()
    
    for type in BlockType.allCases {
        let observation = ValueObservation.tracking(
            region: AnyBlock.select(AnyBlock.SyncableColumns.updatedAt).status(.active).type(type),
            fetch: { db in try AnyBlock.all().status(.active).type(type).fetchAll(db) }
        )
        
        cancellable.append(
            observation.start(in: dbPool) { error in print(error.localizedDescription) }
            onChange: { self.blocks[type] = $0; print("\(Date.now.timeIntervalSince1970): \(type.rawValue) is changed") }
        )
    }
    
}

subscript(_ type: BlockType) -> [AnyBlock] {
    return self.blocks[type] ?? []
}

But the specified regions do not work and I still receive notifications about changes in all block types, despite the fact that I only change one record:

1743579606.406156: section is changed
1743579606.40624: empty is changed
1743579606.40625: group is changed
1743579606.406258: action is changed
...
...
...

How can I receive notifications only from those regions that actually change?

[groue]

Hello @Artem-Viveritsa,

Even when an app defines a very precise region, we're limited by the abilities of SQLite, which can only track columns and rows identified by their numerical rowid. That's it. See DatabaseRegion.

Let's focus on your request:

AnyBlock.select(AnyBlock.SyncableColumns.updatedAt).status(.active).type(type)

I suppose it means "The updatedAt column of all rows of the AnyBlock table that have the status active and the type type".

SQLite can only observe "the updatedAt column of all rows of the AnyBlock table", discarding the status and type filtering.

So GRDB notifies changes even for changes that happen in other statuses and types.

This explains this line of the ValueObservation documentation:

ValueObservation may notify consecutive identical values.

The next sentence is:

You can filter out the undesired duplicates with the removeDuplicates() method.

This is the recommended option when you want to filter out duplicated values.

If you want to better know which database regions are tracked by an observation, use the handleEvents or print methods:

let observation = ValueObservation
    .tracking(
        region: AnyBlock.select(AnyBlock.SyncableColumns.updatedAt).status(.active).type(type),
        fetch: { db in
            try AnyBlock.all().status(.active).type(type).fetchAll(db)
        })
    .handleEvents(willTrackRegion: { region in
        print("tracked region: \(region)")
    }

[Artem-Viveritsa]

Thanks for the reply, @groue! Yes, .removeDuplicates() gives the expected result and the documentation describes it in detail. But if there are such limitations on the part of sqlite, then from the point of view of performance, replacing the array with a new one, even with the same values, seems simpler than comparing them? Provided that each has about 300-2000 elements.

[groue]

But if there are such limitations on the part of sqlite, then from the point of view of performance, replacing the array with a new one, even with the same values, seems simpler than comparing them? Provided that each has about 300-2000 elements.

I agree that it's not always useful to remove duplicates. Personally, I don't really care until I identify an actual problem that requires a practical solution.

[Artem-Viveritsa]

Ok, I think I'll just change the observe logic to account for this feature. Thanks!

[Artem-Viveritsa]

I settled on the following logic, for a 15,000 row database it works fine:

private var allBlocks = Dictionary<UUID, AnyBlock>()

@ObservationIgnored
private var cancellable = [AnyDatabaseCancellable?]()
    
@ObservationIgnored
private var lastSync = Date(timeIntervalSince1970: 0)

private init() { observe() }

private func observe() {
    print("Start observing...")
    
    cancellable.removeAll()

    let observation = ValueObservation.tracking(
        AnyBlock
            .filter(AnyBlock.SyncableColumns.updatedAt > lastSync)
            .order(AnyBlock.SyncableColumns.updatedAt.desc)
            .fetchAll
    )
    
    cancellable.append(
        
        observation.start(in: dbPool.reader) { error in print(error.localizedDescription) }
        
        onChange: { blocks in
            print("New blocks count: \(blocks.count)")
            
            for block in blocks {
                self.allBlocks.updateValue(block, forKey: block.id)
            }
            
            if let latestBlock = blocks.first {
                print("Last sync \(self.lastSync) --> \(latestBlock.updatedAt)")
                self.lastSync = latestBlock.updatedAt
                self.observe()
            }
            
        }
        
    )
    
}

// ~ 1 ms
subscript(_ type: BlockType) -> [AnyBlock] {
    let start = Date.now
    let blocks = self.allBlocks.values.filter { $0.type == type } .sorted { $0.order < $1.order }
    print("\(type.rawValue) filter time: \(String(describing: Date.now.timeIntervalSince(start).formatted(.number)))")
    return blocks
}

subscript(_ id: UUID) -> AnyBlock? {
    return allBlocks[id]
}