[oi-dev] Opening executable with write flags

[Till Wegmüller]

Hi Paul

Have you tried mdb? We have our own debug data format called CTF and 
depending on how you compiled your file the DWARF data may be stripped 
from the binary.

If you want a stacktrace from mdb you can type ::stack

For more detailed usage of mdb check the manpage or ask on #illumos IRC 
or any illumos mailinglist. There are not many illumos devs here on this 
list and I have never used gdb. Only ever mdb.

Greetings
Till


On 14.07.25 11:39, Paul Floyd via oi-dev wrote:
> Hi
> 
> I'm working on an issue related to an exe opening its own binary file 
> with flags like O_WRONLY|O_CREAT|O_TRUNC. Any writing is going to be 
> bad, and truncation even worse.
> 
> On FreeBSD and Linux this is not allowed. 'open' will fail and set errno 
> to ETXTBSY.
> 
> I haven't yet tried this on Solaris.  My testcase just opens and 
> truncates the exe and core dumps on the first call to open.
> 
> Under gdb I get
> 
> Dwarf Error: wrong version in compilation unit header (is 0, should be 
> 2, 3, 4 or 5) [in module /export/home/paulf/scratch/valgrind/memcheck/ 
> tests/solaris/open_client]
> 
> I guess that;s because it's failing to rread the Dwarf from the 0 byte 
> file.
> 
> Here is my testcase, works fine on FreeBSD
> 
> // For Bug 505673
> // Valgrind crashes with an internal error and SIGBUS when the guest 
> tries to open its own file with O_WRONLY|O_CREAT|O_TRUNC
> #include <fcntl.h>
> #include <cerrno>
> #include <stdexcept>
> #include <vector>
> #include <unistd.h>
> #include <sys/syscall.h>
> #include <unistd.h>
> 
> int main(int argc, char** argv)
> {
>      std::vector<int> flags{O_WRONLY|O_CREAT|O_TRUNC, O_WRONLY, O_RDWR};
> 
>      // On FreeBSD libc open uses syscall openat (at least on 14.2)
>      for (auto f : flags)
>      {
>          int res = open(argv[0], f, 0666);
>          if (-1 != res)
>          {
>              throw std::runtime_error("open should have failed");
>          }
>          else
>          {
>              if (errno != ETXTBSY)
>              {
>                  throw std::runtime_error("errno should be ETXTBSY");
>              }
>          }
>      }
> 
>      // repeat the above, but with syscall(SYS_OPEN
>      for (auto f : flags)
>      {
>          int res = syscall(SYS_open, argv[0], f, 0666);
>          if (-1 != res)
>          {
>              throw std::runtime_error("open should have failed");
>          }
>          else
>          {
>              if (errno != ETXTBSY)
>              {
>                  throw std::runtime_error("errno should be ETXTBSY");
>              }
>          }
>      }
> 
>      chdir("..");
> 
>      // check that relative paths work
>      for (auto f : flags)
>      {
>          int res = open("solaris/open_client", f, 0666);
>          if (-1 != res)
>          {
>              throw std::runtime_error("open should have failed");
>          }
>          else
>          {
>              if (errno != ETXTBSY)
>              {
>                  throw std::runtime_error("errno should be ETXTBSY");
>              }
>          }
>      }
> 
>      for (auto f : flags)
>      {
>          int res = syscall(SYS_open, "solaris/open_client", f, 0666);
>          if (-1 != res)
>          {
>              throw std::runtime_error("open should have failed");
>          }
>          else
>          {
>              if (errno != ETXTBSY)
>              {
>                  throw std::runtime_error("errno should be ETXTBSY");
>              }
>          }
>      }
> }
> 
> A+
> 
> Paul
> 
> 
> 
> _______________________________________________
> oi-dev mailing list
> oi-dev at openindiana.org
> https://openindiana.org/mailman/listinfo/oi-dev