Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

omake in linux #1044

Open
mh466lfa opened this issue Sep 9, 2024 · 43 comments
Open

omake in linux #1044

mh466lfa opened this issue Sep 9, 2024 · 43 comments

Comments

@mh466lfa
Copy link

mh466lfa commented Sep 9, 2024

I extracted omake from the orangec source tree (copied all needed source files until the compiler stopped complaining) and added them into a RedPanda DevCpp project. I added -DHAVE_UNISTD_H and -pthread. It compiled fine with a few warnings. But the warning by the linker seems to need attention.

/usr/bin/ld: ./omake/os.o: in function `OS::JobInit()':
os.cpp:(.text+0x98a): warning: the use of `tempnam' is dangerous, better use `mkstemp'
/usr/bin/ld: ./util/Utils.o: in function `Utils::TempName(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&)':
Utils.cpp:(.text+0xd24): warning: the use of `tmpnam' is dangerous, better use `mkstemp'

Anyway, I have a make binary (I named my omake binary simply make).
@mh466lfa
Copy link
Author

mh466lfa commented Sep 9, 2024

The problem is this omake doesn't work with any makefile. It always crashes with this error:

terminate called after throwing an instance of 'std::system_error'
  what():  Bad file descriptor
Aborted

It seems it's looking for make.cfg or unknown.cfg which is not available. On Windows, it works fine without any of these files.

Here is the result of strace:

$ strace ./make
execve("./make", ["./make"], 0x7ffc3c43d880 /* 45 vars */) = 0
brk(NULL)                               = 0x5e9c893ec000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f5adb720000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=78862, ...}, AT_EMPTY_PATH) = 0
mmap(NULL, 78862, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f5adb70c000
close(3)                                = 0
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libstdc++.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\0\0\0\0\0\0\0"..., 832) = 832
newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=2190440, ...}, AT_EMPTY_PATH) = 0
mmap(NULL, 2201728, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f5adb400000
mmap(0x7f5adb499000, 1052672, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x99000) = 0x7f5adb499000
mmap(0x7f5adb59a000, 454656, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x19a000) = 0x7f5adb59a000
mmap(0x7f5adb609000, 57344, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x209000) = 0x7f5adb609000
mmap(0x7f5adb617000, 10368, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f5adb617000
close(3)                                = 0
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\0\0\0\0\0\0\0"..., 832) = 832
newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=125312, ...}, AT_EMPTY_PATH) = 0
mmap(NULL, 127688, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f5adb6ec000
mmap(0x7f5adb6ef000, 94208, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0x7f5adb6ef000
mmap(0x7f5adb706000, 16384, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1a000) = 0x7f5adb706000
mmap(0x7f5adb70a000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1d000) = 0x7f5adb70a000
close(3)                                = 0
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20t\2\0\0\0\0\0"..., 832) = 832
pread64(3, "\6\0\0\0\4\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0"..., 784, 64) = 784
newfstatat(3, "", {st_mode=S_IFREG|0755, st_size=1922136, ...}, AT_EMPTY_PATH) = 0
pread64(3, "\6\0\0\0\4\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0"..., 784, 64) = 784
mmap(NULL, 1970000, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f5adb21f000
mmap(0x7f5adb245000, 1396736, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x26000) = 0x7f5adb245000
mmap(0x7f5adb39a000, 339968, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17b000) = 0x7f5adb39a000
mmap(0x7f5adb3ed000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1ce000) = 0x7f5adb3ed000
mmap(0x7f5adb3f3000, 53072, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f5adb3f3000
close(3)                                = 0
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libm.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\0\0\0\0\0\0\0"..., 832) = 832
newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=907784, ...}, AT_EMPTY_PATH) = 0
mmap(NULL, 909560, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f5adb140000
mmap(0x7f5adb150000, 471040, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x10000) = 0x7f5adb150000
mmap(0x7f5adb1c3000, 368640, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x83000) = 0x7f5adb1c3000
mmap(0x7f5adb21d000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xdc000) = 0x7f5adb21d000
close(3)                                = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f5adb6ea000
mmap(NULL, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f5adb6e7000
arch_prctl(ARCH_SET_FS, 0x7f5adb6e7740) = 0
set_tid_address(0x7f5adb6e7a10)         = 3777
set_robust_list(0x7f5adb6e7a20, 24)     = 0
rseq(0x7f5adb6e8060, 0x20, 0, 0x53053053) = 0
mprotect(0x7f5adb3ed000, 16384, PROT_READ) = 0
mprotect(0x7f5adb21d000, 4096, PROT_READ) = 0
mprotect(0x7f5adb70a000, 4096, PROT_READ) = 0
mprotect(0x7f5adb609000, 45056, PROT_READ) = 0
mprotect(0x5e9c6c8e2000, 4096, PROT_READ) = 0
mprotect(0x7f5adb758000, 8192, PROT_READ) = 0
prlimit64(0, RLIMIT_STACK, NULL, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
munmap(0x7f5adb70c000, 78862)           = 0
getrandom("\x69\x82\x0a\xe3\x27\x95\x24\x56", 8, GRND_NONBLOCK) = 8
brk(NULL)                               = 0x5e9c893ec000
brk(0x5e9c8940d000)                     = 0x5e9c8940d000
futex(0x7f5adb61773c, FUTEX_WAKE_PRIVATE, 2147483647) = 0
access("./make.cfg", F_OK)              = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "unknown.cfg", O_RDONLY) = -1 ENOENT (No such file or directory)
pipe2([3, 4], 0)                        = 0
write(-1, "1", 1)                       = -1 EBADF (Bad file descriptor)
futex(0x7f5adb70b1f0, FUTEX_WAKE_PRIVATE, 2147483647) = 0
write(2, "terminate called after throwing "..., 48terminate called after throwing an instance of ') = 48
write(2, "std::system_error", 17std::system_error)       = 17
write(2, "'\n", 2'
)                      = 2
write(2, "  what():  ", 11  what():  )             = 11
write(2, "Bad file descriptor", 19Bad file descriptor)     = 19
write(2, "\n", 1
)                       = 1
rt_sigprocmask(SIG_UNBLOCK, [ABRT], NULL, 8) = 0
gettid()                                = 3777
getpid()                                = 3777
tgkill(3777, 3777, SIGABRT)             = 0
--- SIGABRT {si_signo=SIGABRT, si_code=SI_TKILL, si_pid=3777, si_uid=1000} ---
+++ killed by SIGABRT +++
Aborted

@GitMensch
Copy link
Contributor

Please move that to https://github.com/LADSoft/OMAKE, which should also make it easy to build without manual deletion of files.

@chuggafan
Copy link
Contributor

chuggafan commented Sep 9, 2024 via email

@mh466lfa
Copy link
Author

Please move that to https://github.com/LADSoft/OMAKE, which should also make it easy to build without manual deletion of files.

This repository is not synced with the latest changes. Getting your build system to work on Linux is also a hassle. This is why I decided to use my own build system (a RedPanda DevCpp project).

@mh466lfa
Copy link
Author

mh466lfa commented Sep 10, 2024 •
edited
Loading

@chuggafan I changed line 110 in ToolChain.cpp from if (access(configName.c_str(), 0) != 0) to if (access(configName.c_str(), 0) != F_OK) but it doesn't help. I'm willing to test your changes. Please tell me where could I clone your code.

@mh466lfa
Copy link
Author

mh466lfa commented Sep 10, 2024 •
edited
Loading

Note: I build omake completely decoupled from orange c. ToolChain::StandardToolStartup makes a reference to the ORANGEC environment variable. It seems omake is tied to orange c? I don't have orange c. Building omake alone in linux is easy. But the whole orange c suite is another story.

@GitMensch
Copy link
Contributor

Please move that to https://github.com/LADSoft/OMAKE, which should also make it easy to build without manual deletion of files.

This repository is not synced with the latest changes.

Right. That repo's state is considered to be stable and has an issue tracker specific for omake, while this repo's master is both focused on the whole suite and on "latest testing" state.

Note: I build omake completely decoupled from orange c. ToolChain::StandardToolStartup makes a reference to the ORANGEC environment variable. It seems omake is tied to orange c?

That's a reasonable thing for the defaults - the builtin rules need an internal define for CC/CXX/AS (= if not explicit specified in the makefile) and those use the OrangeC tools; their fullpath is resolved from the ORANGEC environment variable and if not already set this is done internally, relative from omakes path.

@chuggafan
Copy link
Contributor

chuggafan commented Sep 11, 2024 via email

@mh466lfa
Copy link
Author

mh466lfa commented Sep 11, 2024 •
edited
Loading

Also, renaming it when you should have omake.cfg will cause it to hunt for
make.cfg for the check, so that's another weird thing...

@chuggafan Please have a look at your ToolChain::StandardToolStartup method:

https://github.com/LADSoft/OrangeC/blob/master/src/util/ToolChain.cpp#L99

The name of the .cfg file will be the name of the make binary + .cfg. I name my omake binary make, so the .cfg file is make.cfg. Nothing weird. unknown.cfg should be somewhere in the code. Maybe when it can't determine the current make binary name, it will use unknown. I don't know for sure. I'm not dive into the code yet.

@mh466lfa
Copy link
Author

I've done some investigation, it seems to be something up with my jobserver implementation, I only did a preliminary investigation and I'll do a further one over the weekend but it seems like my "quick fix" is actually "wtf am I doing on Unix again?" Probably from not initializing job checks, we'll see...

The problem is I only use just one make job (doesn't specify -j at all). It seems you are going the wrong direction. Compiling it with debug and running it in gdb will reveal interesting things. Everything points to the BUILTINS in MakeMain.cpp and thus the .cfg file and the ToolChain::StandardToolStartup method.

@GitMensch
Copy link
Contributor

GitMensch commented Sep 11, 2024 via email

@mh466lfa
Copy link
Author

I managed to compile the whole orange c suite in linux. This is the command I used:

make COMPILER="gcc-linux"

But I don't know what to do after that. How to install it so I could set the ORANGEC environment variable?

Btw, I noticed that the binaries have .exe extensions even though it's Linux. I checked using file and they are indeed Linux ELF binaries.

@mh466lfa
Copy link
Author

Frankly speaking, I'm more interested in omake as a standalone tool. If I need the whole orange c suite to use omake then it's a lot less appealing to me.

@mh466lfa
Copy link
Author

Please move that to https://github.com/LADSoft/OMAKE, which should also make it easy to build without manual deletion of files.

This repository is not synced with the latest changes.

Right. That repo's state is considered to be stable and has an issue tracker specific for omake, while this repo's master is both focused on the whole suite and on "latest testing" state.

The omake in this repository will not even compile with make COMPILER="gcc-linux". It's full of compilation errors.

@mh466lfa
Copy link
Author

Mr. ghost (#1038) is absolutely right. Please run static analyzers on your code. Or at least try compiling using latest GCC and Clang with the highest diagnostics. Goodbye.

@GitMensch
Copy link
Contributor

GitMensch commented Sep 12, 2024 via email

@mh466lfa mh466lfa reopened this Sep 12, 2024
@mh466lfa
Copy link
Author

Thanks for the note, but the issue should be kept open as it is unresolved (please opt out of subscription instead if you don't want to receive any updates).

OK.

@chuggafan
Copy link
Contributor

chuggafan commented Sep 14, 2024 via email

@GitMensch
Copy link
Contributor

Please consider MSYS and possibly cygwin - using a test for the shell may break some parts if that's for finding the OS...

You do know about the OS environment variable, right?

@chuggafan
Copy link
Contributor

chuggafan commented Sep 14, 2024 via email

@GitMensch
Copy link
Contributor

GitMensch commented Sep 14, 2024 via email

@LADSoft
Copy link
Owner

LADSoft commented Sep 14, 2024

@chuggafan thank you for looking into this.

FWIW I've been trying to get away from checking the _WIN32 define and I made a define TARGET_OS_WINDOWS... the intention was when that isn't set it is for linux/unix but I don't suppose there is any reason we couldn't additionally add TARGET_OS_LINUX if that seems reasonable....

@chuggafan
Copy link
Contributor

chuggafan commented Sep 14, 2024 via email

@chuggafan
Copy link
Contributor

chuggafan commented Sep 15, 2024 via email

@chatchoi
Copy link

Look at this for a reference on POSIX system (how to spawn new processes, environment variables, blah blah...):

https://github.com/rmyorston/pdpmake

Btw, I suggest using POSIX instead of Linux. You don't want your software to run on other *nixes like FreeBSD?

@chatchoi
Copy link

Your code is GPL-ed, and since the GPL can absorb code written in weaker licenses, I don't see there are any reasons to not look into the code of GNU make itself or BSD make and even copy it.

bmake is here: https://www.crufty.net/help/sjg/bmake.html

@chatchoi
Copy link

chatchoi commented Sep 15, 2024 •
edited
Loading

uname woes on MSYS2: https://codeberg.org/schilytools/schilytools/issues/60

Note: CDDL is not compatible with GPL.

@chatchoi
Copy link

Please consider MSYS and possibly cygwin - using a test for the shell may break some parts if that's for finding the OS...

You do know about the OS environment variable, right?

Orange C failed to link on Cygwin and FreeBSD. Compilation is fine, but linking is not. Why?

@chatchoi
Copy link

And as @mh466lfa has noted, the executables have .exe extensions even on Linux and FreeBSD.

@chatchoi
Copy link

And why -D__MSVCRT__ is there even on Linux and FreeBSD?

@chatchoi
Copy link

chatchoi commented Sep 15, 2024 •
edited
Loading

It's linking fine on Linux, but failed to link on Cygwin and FreeBSD.

On Cygwin:

g++ -c -D__MSVCRT__ -DHAVE_UNISTD_H=1 -DSQLITE_OS_UNIX -Wno-int-to-pointer-cast -I../util -I../ocpp -I../occ -I../objlib -I../oasm -I../occopt -fpermissive -Dx64 -DUSE_LONGLONG -DGNUC -std=gnu++14 -o /c/OrangeC/src/occopt/obj/gcc-linux/optmain.o optmain.cpp
gcc -L/c/OrangeC/src/../src/lib/gcc-linux -o occopt.exe /c/OrangeC/src/occopt/obj/gcc-linux/optmain.o -Wl,--start-group -loccopt -lutil -locpplib -Wl,--end-group -lstdc++ -lpthread -ldl
/usr/lib/gcc/x86_64-pc-msys/13.3.0/../../../../x86_64-pc-msys/bin/ld: /c/OrangeC/src/../src/lib/gcc-linux/liboccopt.a(iconst.o):iconst.cpp:(.text+0x5b4a): undefined reference to `Optimizer::RemoveInstruction(Optimizer::quad_t*)'
collect2: error: ld returned 1 exit status

On FreeBSD, it's complaining about missing of ceilf as I recall.

@chatchoi
Copy link

chatchoi commented Sep 15, 2024 •
edited
Loading

Here is the error message on FreeBSD:

clang++ -c -D__MSVCRT__ -DHAVE_UNISTD_H=1 -DSQLITE_OS_UNIX -Wno-int-to-pointer-cast -I../util -I../exefmt -I../ -fpermissive  -DGNUC -std=gnu++14 -o /test/OrangeC/src/netlib/obj/gcc-linux/NetLinkMain.o NetLinkMain.cpp
clang -L/test/OrangeC/src/../src/lib/gcc-linux -o netlib.exe /test/OrangeC/src/netlib/obj/gcc-linux/NetLinkMain.o -Wl,--start-group -lnetlib -lutil -Wl,--end-group -lstdc++ -lpthread -ldl 
ld: error: undefined symbol: ceilf
>>> referenced by NetLinkMain.cpp
>>>               /test/OrangeC/src/netlib/obj/gcc-linux/NetLinkMain.o:(std::__1::__math::ceil[abi:se180100](float))
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Build command: gmake COMPILER="gcc-linux" CC=clang CXX=clang++.

@chuggafan
Copy link
Contributor

chuggafan commented Sep 15, 2024 via email

@chatchoi
Copy link

Lack of the ceilf function indicates to me that one needs to add the -lm linker flag to link libmath on those platforms for everything to build.

What about Cygwin? Don't tell me you only read the last email. Please login to your Github account to read the whole thread. Github is more like a social network or a forum. You can't work with it optimally if you treat it as a mailing list. You have conveniently ignored other things posted by me.

@chatchoi
Copy link

It's either pure carelessness (only read the last email) or intentional. Because even as it non-optimal form (a mailing list), Github does send all of the messages people have posted to everyone subscribed to the thread.

@chatchoi
Copy link

If a big thread is too difficult for you to keep track of, I will split my comments into new issues.

@GitMensch GitMensch mentioned this issue Sep 16, 2024
Open
@chuggafan
Copy link
Contributor

chuggafan commented Sep 16, 2024 via email

@chatchoi
Copy link

I'm also testing your code for free. I need better cooperation. I don't like to talk alone.

Btw, unsubscribed. Goodbye.

@chuggafan
Copy link
Contributor

chuggafan commented Nov 3, 2024 via email

@chuggafan
Copy link
Contributor

Ya'know what, re-reading how GNU make does it, it turns out I'm an idiot.
https://www.gnu.org/software/make/manual/html_node/Execution.html
GNU Make (at least for non-.ONESHELL) just executes system.
We can get away with posix_spawn fairly easily if we want to pass in a custom environment by just ensuring the first two parameters are:

  1. whatever $(SHELL) is.
  2. -c
  3. the entire line we're running.
    It's that easy unless we care about .ONESHELL
    Why am I stupid?

@chuggafan
Copy link
Contributor

Currently working on this since my revelation, currently trying to understand duplicating pipes since there's a way to duplicate a pipe using posix_spawn so that I can read it for the $(system) construction..., just going to have to power through, should have something before next month if I continue to steady-state work on this with research.

@GitMensch
Copy link
Contributor

next month = next year = ... 24 hours :-)
I suppose you mean before end of January 2025.

@chuggafan
Copy link
Contributor

Yes, of course, I'll probably throw my work up over the next weekend just to get some eyeballs on the $(system) part since I'm utterly confused about where to go on that...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@LADSoft @GitMensch @chuggafan @mh466lfa @chatchoi