Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

using MemoryMappedFiles on non-windows systems? #29

Open
pracplayopen opened this issue May 3, 2024 · 2 comments
Open

using MemoryMappedFiles on non-windows systems? #29

pracplayopen opened this issue May 3, 2024 · 2 comments

Comments

@pracplayopen
Copy link

Tried to accomplish this using the example here:

AnalyzeTicks.cs

However this generated a DllNotFoundException for kernel32.
Kernel32 is a windows-only thing, looked into source a bit:

public static unsafe RawMemoryMapping<T> OpenRead(string fileName)
    {
      if (fileName == null)
        throw new ArgumentNullException(nameof (fileName));
      RawMemoryMapping<T> rawMemoryMapping = new RawMemoryMapping<T>();
      rawMemoryMapping.teaFile = TeaFile<T>.OpenRead(fileName);
      try
      {
        if (!(rawMemoryMapping.teaFile.Stream is FileStream stream))
          throw new InvalidOperationException("TeaFile used for MemoryMapping is not a file stream but memory mapping requires a file.");
        rawMemoryMapping.mappingHandle = UnsafeNativeMethods.CreateFileMapping(stream.SafeFileHandle, IntPtr.Zero, MapProtection.PageReadOnly, 0, 0, (string) null);
        if (rawMemoryMapping.mappingHandle.IsInvalid)
          throw new Win32Exception();
        try
        {
          rawMemoryMapping.mappingStart = UnsafeNativeMethods.MapViewOfFile(rawMemoryMapping.mappingHandle, MapAccess.FileMapRead, 0, 0, IntPtr.Zero);
          if ((IntPtr) rawMemoryMapping.mappingStart == IntPtr.Zero)
            throw new Win32Exception();
          rawMemoryMapping.itemAreaStart = rawMemoryMapping.mappingStart + rawMemoryMapping.teaFile.ItemAreaStart;
          rawMemoryMapping.itemAreaEnd = rawMemoryMapping.itemAreaStart + rawMemoryMapping.teaFile.ItemAreaSize;
        }
        catch
        {
          rawMemoryMapping.mappingHandle = UnsafeNativeMethods.CloseHandle(rawMemoryMapping.mappingHandle) != 0 ? (SafeFileHandle) null : throw new Win32Exception();
          throw;
        }
        return rawMemoryMapping;
      }
      catch
      {
        rawMemoryMapping.Dispose();
        throw;
      }
    }

  internal static class UnsafeNativeMethods
  {
    [DllImport("kernel32", CharSet = CharSet.Unicode, SetLastError = true)]
    public static extern SafeFileHandle CreateFileMapping(
      SafeFileHandle hFile,
      IntPtr lpAttributes,
      MapProtection flProtect,
      int dwMaximumSizeHigh,
      int dwMaximumSizeLow,
      string lpName);

//...
}

So it's the calls inside this UnsafeNativeMethods which won't work as implemented.

In recent dotnets, is it possible to reimplement UnsafeNativeMethods using MemoryMappedFile class?

Not all of MemoryMappedFile classes methods will run outside of windows, but MemoryMappedFile.CreateFromFile does work.

Any ideas or suggestions on how this might be accomplished?

@thulka
Copy link
Member

thulka commented May 3, 2024

You are right, for Linux, Unix, MacOS we need a dotnet MemoryMapping class that accesses the classic unix mmap api.

I do not believe that the recent dotnets change this necessity, the dotnet BCL tools for memory mapping were never as efficient as this raw access we use here. Have not checked if that changed meanwhile.

You can write (and contribute) a raw mmap access for unix and most likely you will find a template here at github.
Searching for "mmap munmap language:c#"

and then filter on source files gives for instance

https://github.com/ProjectMagma/Magma/blob/3ba146cc864edfe8fbb8affd66c448cdaa69248e/src/Magma.NetMap/Interop/Libc/MMap.cs#L10

@pracplayopen
Copy link
Author

thanks this is helpful
seems pretty interesting maybe able to talk somebody into it and will take all your suggestions into account
hopefully we can leave the issue open just to make it easy to find, since it is something anybody could help to advance the cause

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants