Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor: Using only DirectX and ImageSharp based bitmap processing over GDI #548

Merged
merged 1 commit into from
Nov 5, 2023

Conversation

Xian55
Copy link
Owner

@Xian55 Xian55 commented Nov 5, 2023

Changes:

  • In computation heavy part of the code replace System.Drawing with ImageSharp library.
  • Replaced most of the System.Drawing.Bitmap references in favour of ImageSharp.Image<Bgra32>
  • Removed GDI based AddonDataProvider
  • Most image processing uniformly uses ImageSharp lib, expect Mouse bitmap.

Reasoning:
Over the past few weeks i've been experimented with only using DXGI in the project, finding a way to uniformly extract the texture from GPU memory to the main memory, on the other hand processing these textures more efficiently.

While System.Drawing.Bitmap works really well, it has a really low processing latency, but the grabbing a frame operation its rather slow and block the thread, in a single point in time, there can be one and only lock on it, so multithreading is out of the question. On top of that, it actually used double locking with the LockBits API. I've made a workaround by using lock all the time but it didn't felt right. Currently there are a few threads which would like to access the bitmap data only read-only mode.

ImageSharp comes to the rescue with a modern Span<T>/Memory<T> based based API, leveraging all the modern features what dotnet6 can provide. It supports single writer multiple reader mechanism for working with Bitmap based data.

DXGI screen capture over GDI shows really good performance boost, with greatly reduced latency.

All numbers are in milliseconds timespans

// GDI
// Graphics.CopyFromScreen
// When the API is called, this is the duration when the next frame is produced
// and the ExecutionContext returns
capture | sample: 4861 | avg: 18.30 | min: 14.94 | max: 029.385 | total: 88946.80 | std: 010.40 | thres: 29.4233ms

// DXGI 
// DirectX based DXGIDesktopDuplicator.AcquireNextFrame
// with the timeoutInMilliseconds parameter there's an option to block the thread or return
// When there is no new Frame the API simply returns
// Only shows the Buffer.MemoryCopy time
capture | sample: 4764 | avg: 0.19 | min: 0.02 | max: 001.059 | total: 882.16 | std: 000.81 | thres: 1.0611ms

// DXGI is capable of producing new frames roughly around every 10ms

In the future i plan to take advantage of ImageSharp, executing NpcNameFinder and MinimapNodeFinder
code at the GPU side.

…g with ImageSharp library.

Removed AddonDataProviderGDI
@Xian55 Xian55 added refactor This ticket concerns the possible simplification of code/data. enhancement This pull request implements a new feature. breaking change labels Nov 5, 2023
@Xian55 Xian55 changed the title Refactor: Using only DirectX and ImageSharp based bitmap processing over GDI Refactor: Using only DirectX and ImageSharp based bitmap processing over GDI Nov 5, 2023
@Xian55 Xian55 merged commit 73929a7 into dev Nov 5, 2023
1 check passed
@Xian55 Xian55 deleted the refactor/system-drawing-mostly-replaced-by-imagesharp branch November 5, 2023 03:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking change enhancement This pull request implements a new feature. refactor This ticket concerns the possible simplification of code/data.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant