File not found after prioritizing a contiguous one from a previous session

[YakoYakoYokuYoku]

Please provide the following information

libtorrent version (or branch): RC_2_0

platform/architecture: Linux 6.12.9

compiler and compiler version: Clang 19.1.7

Continuing testing my GStreamer BitTorrent streaming plugin I've noticed when prioritizing for only one torrent member to be downloaded, by setting lt::top_priority for that one and lt::dont_download for the rest, and saving its last piece to disk that if I try to download its next contiguous file in another session libtorrent will report that such file is not found because it only wrote to disk the prioritized bebmber. Said in another manner, downloading the last piece of a prioritized file which contains data from its adyacent will confound libtorrent into erroneously thinking that it has the piece in its entirety.

Such error doesn't happen all the times, it's a bit unpredictable but I've stumbled upon it.

I do have a pair of ideas on how to fix it, either libtorrent detects that the piece has been partially saved and downloads it again or it could write the remaining data into a special file which contains piece fragments. I'm inclined towards the latter to avoid redundant but the former has the advantage of rechecking things again for sanity.

[arvidn]

Said in another manner, downloading the last piece of a prioritized file which contains data from its adyacent will confound libtorrent into erroneously thinking that it has the piece in its entirety.

I don't quite understand all the specifics of this issue. Let's say you have a torrent with 2 files, A and B. You download A and set B to dont_download. Assuming the last piece of A overlaps with the beginning of file B; once that piece has been completely downloaded, part of it will be saved to file B, right? Or is it being saved to the part file?

Your statement first states that the last piece has been downloaded, then you suggest that it hasn't been downloaded "in its entirety". Which one is it? pieces can only be verified against the piece hash if the whole piece has been downloaded.

I think the next step in trouble shooting is to understand whether file B exists on disk or not. If it doesn't, do you have a part-file? (typically named by the info-hash and .parfile). If so, the way to have libtorrent pick up the part file when adding the torrent back is to make sure you set the file priorities to the same as they were when you stopped the previous session.

If you save resume data, in and add_torrent_params object, it should already include the file priorities, and you can just add the torrent back by using it.

[YakoYakoYokuYoku]

I do have a pair of ideas on how to fix it, either libtorrent detects that the piece has been partially saved and downloads it again or it could write the remaining data into a special file which contains piece fragments.

Errata, this feature is already present in the form of part files. An oversight from my side.

Assuming the last piece of A overlaps with the beginning of file B; once that piece has been completely downloaded, part of it will be saved to file B, right? Or is it being saved to the part file?

I think that the portion of B it's being saved into the part file per what a hex dump on a .parts of the torrent shows (I've looked for EBML headers of Matroska).

Your statement first states that the last piece has been downloaded, then you suggest that it hasn't been downloaded "in its entirety". Which one is it? pieces can only be verified against the piece hash if the whole piece has been downloaded.

Maybe I wasn't that clear on that. I've referred to download as fetching the piece and writing it into a corresponding file. And again, what I wrote was because I've forgot that .parts existed.

If so, the way to have libtorrent pick up the part file when adding the torrent back is to make sure you set the file priorities to the same as they were when you stopped the previous session.

That makes me think, does libtorrent use the part file when you change said priorities? I've assumed that .parts is checked regardless of that.

If you save resume data, in and add_torrent_params object, it should already include the file priorities, and you can just add the torrent back by using it.

My plugin uses resume data for convenience. Though, priorities have to be changed when I select another torrent member because said plugin works on a single file as of now. If that changes the way the part file is read then I wasn't aware of that.