[oi-dev] problems publishing rust

Mon Jun 20 19:21:58 UTC 2022

Am 20.06.22 um 21:18 schrieb Friedrich Kink via oi-dev:
> Hi Till,
>
> thanks a lot this is indeed a very valuable hint. Unfortunately since
> a couple of compile runs installation fails at the very end with the
> following error:
>
> ...
>
> Install analysis stage2 (Some(TargetSelection { triple:
> "x86_64-unknown-illumos", file: None }))
> install: creating uninstall script at
> /usr/src/myoi-userland/components/developer/rust/build/prototype/i386/usr/lib/rustlib/uninstall.sh
> install: installing component 'rust-analysis-x86_64-unknown-illumos'
>
>     rust analysis installed.
>
> Dist rust-src-1.61.0
> Error: failed to generate installer
>
> Caused by:
>     0: failed to tar file
> '/usr/share/src/myoi-userland/components/developer/rust/build/amd64/build/tmp/tarball/rust-src/rust-src-1.61.0/rust-src/lib/rustlib/src/rust/library/portable-simd/LICENSE-APACHE'
>     1: provided value is too long when setting link name for
We already had another guy trying to build a newer rust failing with the
same problem.
He found out that this is a known problem and there should already be a
fix (as far as I understood) but it hadn't been integrated in the release.
It looks like the situation didn't change in the last weeks.


>
> Stack backtrace:
>    0: <anyhow::Error as core::convert::From<std::io::error::Error>>::from
>    1:
> installer::tarballer::append_path::<std::io::buffered::bufwriter::BufWriter<alloc::boxed::Box<dyn
> installer::compression::Encoder>>>
>    2:
> <rayon_core::thread_pool::ThreadPool>::install::<<installer::tarballer::Tarballer>::run::{closure#2},
> core::result::Result<(), anyhow::Error>>::{closure#0}
>    3:
> <core::panic::unwind_safe::AssertUnwindSafe<<rayon_core::job::StackJob<_,
> _, _> as rayon_core::job::Job>::execute::call<core::result::Result<(),
> anyhow::Error>,
> <rayon_core::registry::Registry>::in_worker_cold<<rayon_core::thread_pool::ThreadPool>::install<<installer::tarballer::Tarballer>::run::{closure#2},
> core::result::Result<(), anyhow::Error>>::{closure#0},
> core::result::Result<(),
> anyhow::Error>>::{closure#0}::{closure#0}>::{closure#0}> as
> core::ops::function::FnOnce<()>>::call_once
>    4: <rayon_core::job::StackJob<&rayon_core::latch::LockLatch,
> <rayon_core::registry::Registry>::in_worker_cold<<rayon_core::thread_pool::ThreadPool>::install<<installer::tarballer::Tarballer>::run::{closure#2},
> core::result::Result<(), anyhow::Error>>::{closure#0},
> core::result::Result<(), anyhow::Error>>::{closure#0}::{closure#0},
> core::result::Result<(), anyhow::Error>> as
> rayon_core::job::Job>::execute
>    5: <rayon_core::registry::WorkerThread>::wait_until_cold
>    6: <rayon_core::registry::ThreadBuilder>::run
>    7:
> std::sys_common::backtrace::__rust_begin_short_backtrace::<<rayon_core::registry::DefaultSpawn
> as rayon_core::registry::ThreadSpawn>::spawn::{closure#0}, ()>
>    8:
> <<std::thread::Builder>::spawn_unchecked_<<rayon_core::registry::DefaultSpawn
> as rayon_core::registry::ThreadSpawn>::spawn::{closure#0},
> ()>::{closure#1} as
> core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
>    9: <alloc::boxed::Box<F,A> as
> core::ops::function::FnOnce<Args>>::call_once
>              at
> /rustc/fe5b13d681f25ee6474be29d748c65adcd91f69e/library/alloc/src/boxed.rs:1861:9
>   10: <alloc::boxed::Box<F,A> as
> core::ops::function::FnOnce<Args>>::call_once
>              at
> /rustc/fe5b13d681f25ee6474be29d748c65adcd91f69e/library/alloc/src/boxed.rs:1861:9
>   11: std::sys::unix::thread::Thread::new::thread_start
>              at
> /rustc/fe5b13d681f25ee6474be29d748c65adcd91f69e/library/std/src/sys/unix/thread.rs:108:17
>   12: _thrp_setup
>   13: <unknown>
> Build completed unsuccessfully in 0:26:51
> make: *** [/usr/share/src/myoi-userland/make-rules/configure.mk:195:
> /usr/src/myoi-userland/components/developer/rust/build/amd64/.installed]
> Error 1
>
> to me a the path length looks still ok (< 256). So any one any idea? I
> don't know why it happens all of a sudden as I changed nothing basic
> to the environment.
>
> kind regards,
>
>   Fritz
>
> Am 20.06.2022 um 15:23 schrieb Till Wegmueller:
>> Hey Fritz
>>
>> That must be a syntax error in the Makefile. The resolver logic is
>> built into OI-userland and should not get impacted by modifications
>> to Makefile of a component. However for some reason 'RESOLVE_DEPS='
>> leaks into the final filepath. That should not happen.
>>
>> Looking at you Makefile I think the culprit are the lines after
>> '#COMPONENT_POST_INSTALL_ACTION= \' IIRC they need to be commented
>> out too as they have a backslash thus parsing weirdness starts to
>> happen.
>>
>> Hope this helps
>> Greetings
>> Till
>>
>> On 19/06/2022 06.05, Friedrich Kink via oi-dev wrote:
>>> Thanks a lot for your response. Let me give some more information.
>>> This patch helped me to over come your issue:
>>>
>>> --- rustc-1.61.0-src/src/bootstrap/builder.rs   2022-05-18
>>> 03:29:36.000000000 +0000
>>> +++ rustc-1.61.0-src/src/bootstrap/builder.rs.new 2022-06-06
>>> 21:25:45.179276851 +0000
>>> @@ -1304,7 +1304,7 @@
>>>                   Some("-Wl,-rpath, at loader_path/../lib")
>>>               } else if !target.contains("windows") {
>>>                   rustflags.arg("-Clink-args=-Wl,-z,origin");
>>> -                Some("-Wl,-rpath,$ORIGIN/../lib")
>>> +               Some("-Wl,-rpath,/usr/clang/13.0/lib")
>>>               } else {
>>>                   None
>>>               };
>>>
>>> Also I already use rustup (as suggested by Joshua, too) and this is
>>> my current Makefile (as metioned it gets compiled and installed but
>>> make REQUIRED_PACKAGES|publish stops immediately):
>>>
>>> BUILD_BITS=             64
>>> USE_OPENSSL11=          yes
>>>
>>> include ../../../make-rules/shared-macros.mk
>>>
>>> COMPONENT_NAME=         rustc
>>> COMPONENT_VERSION=      1.61.0
>>> COMPONENT_FMRI=         developer/lang/rustc
>>> COMPONENT_SUMMARY=      Rust - Safe, concurrent, practical language
>>> COMPONENT_CLASSIFICATION=       Development/Other Languages
>>> COMPONENT_PROJECT_URL=  http://www.rust-lang.org
>>> COMPONENT_SRC= $(COMPONENT_NAME)-$(COMPONENT_VERSION)-src
>>> COMPONENT_ARCHIVE=      $(COMPONENT_SRC).tar.gz
>>> COMPONENT_ARCHIVE_HASH=
>>> sha256:ad0b4351675aa9abdf4c7e066613bd274c4391c5506db152983426376101daed
>>> COMPONENT_ARCHIVE_URL=
>>> https://static.rust-lang.org/dist/$(COMPONENT_ARCHIVE)
>>> COMPONENT_LICENSE=      MIT or Apache-2.0
>>> COMPONENT_LICENSE_FILE= LICENSE-APACHE
>>>
>>> RUST_BOOTSTRAP_PATH=    $(BUILD_DIR)/$(MACH64)
>>> #RUST_ARCH=             x86_64-sun-solaris
>>>
>>> include $(WS_MAKE_RULES)/common.mk
>>>
>>> # GCC_VERSION =  7
>>>
>>> # Put the bits cargo downloads in a private directory.  This could
>>> be cached
>>> # somewhere more permanent, but it's important to make sure that a
>>> person's
>>> # $HOME/.cargo isn't used.
>>> RUST_VERSION=           $(COMPONENT_VERSION)
>>> RUSTUP_HOME=            $(BUILD_DIR)/$(MACH64)
>>> CARGO_HOME=             $(BUILD_DIR)/$(MACH64)
>>>
>>> PATH= $(CARGO_HOME)/bin:$(GCC_BINDIR):$(PATH.gnu)
>>>
>>> #  workaround the symlink name lenght 100 problem, but hope it is
>>> not neccessary with
>>> # but also with patch src_tools_rust-installer_src_generator.rs.patch
>>> COMPONENT_BUILD_ENV+=   CARGO_HOME=$(CARGO_HOME)
>>> COMPONENT_INSTALL_ENV+= CARGO_HOME=$(CARGO_HOME)
>>> COMPONENT_TEST_ENV+=    CARGO_HOME=$(CARGO_HOME)
>>> COMPONENT_BUILD_ENV+=   RUSTUP_HOME=$(RUSTUP_HOME)
>>> COMPONENT_INSTALL_ENV+= RUSTUP_HOME=$(RUSTUP_HOME)
>>> COMPONENT_TEST_ENV+=    RUSTUP_HOME=$(RUSTUP_HOME)
>>> #       $(COMPONENT_DIR)/files/sym2hard.py; \
>>>
>>> COMPONENT_PRE_CONFIGURE_ACTION += ( \
>>>          $(CLONEY) $(SOURCE_DIR) $(@D); \
>>>          cd $(@D); \
>>>          export RUSTUP_HOME=$(RUSTUP_HOME); \
>>>          export CARGO_HOME=$(CARGO_HOME); \
>>>          export RUSTUP_INIT_SKIP_PATH_CHECK=yes; \
>>>          curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs |
>>> sh -s -- --no-modify-path --default-toolchain stable -y; \
>>>          source $(CARGO_HOME)/env; \
>>>          rustup install stable; \
>>>          rustup default stable; \
>>>          rustup show; \
>>>          cargo install --root=$(CARGO_HOME) cbindgen; \
>>>          );
>>>
>>> PYTHON_VERSION=         3.9
>>>
>>> # Need some help to pick the correct ar binary
>>> GNUAR=$(GNUBIN)/ar
>>> # getpwuid_r failure
>>> CXXFLAGS += -D_POSIX_PTHREAD_SEMANTICS
>>> CFLAGS += -D_POSIX_PTHREAD_SEMANTICS
>>>
>>> # Add arch triplet to pkg macros
>>> #PKG_MACROS+= RUST_ARCH="$(RUST_ARCH)"
>>>
>>> # Rust expects the library path to be /usr/lib and is broken otherwise
>>> RUSTC_LIBDIR= $(USRLIBDIR)
>>> CLANG_ROOT= /usr/clang/13.0
>>>
>>> CONFIGURE_OPTIONS = --prefix=$(CONFIGURE_PREFIX)
>>> CONFIGURE_OPTIONS += --mandir=$(CONFIGURE_MANDIR)
>>> CONFIGURE_OPTIONS += --bindir=$(CONFIGURE_BINDIR.$(BITS))
>>> CONFIGURE_OPTIONS += --libdir=$(RUSTC_LIBDIR)
>>> CONFIGURE_OPTIONS +=
>>> --docdir=$(USRSHAREDOCDIR)/rust-$(COMPONENT_VERSION)
>>> CONFIGURE_OPTIONS += --datadir=$(USRSHAREDIR)
>>> CONFIGURE_OPTIONS += --sysconfdir=$(ETCDIR)
>>> CONFIGURE_OPTIONS += --local-rust-root=$(RUST_BOOTSTRAP_PATH)
>>> CONFIGURE_OPTIONS += --enable-extended       # Build and install
>>> cargo too.
>>> CONFIGURE_OPTIONS += --default-linker=$(CC)
>>> CONFIGURE_OPTIONS += --set rust.default-linker=$(CC)
>>> CONFIGURE_OPTIONS += --enable-rpath
>>> CONFIGURE_OPTIONS += --disable-codegen-tests
>>> CONFIGURE_OPTIONS += --disable-dist-src
>>> CONFIGURE_OPTIONS += --disable-llvm-static-stdcpp
>>> CONFIGURE_OPTIONS += --enable-ninja
>>> CONFIGURE_OPTIONS += --enable-vendor
>>> CONFIGURE_OPTIONS += --disable-docs
>>> CONFIGURE_OPTIONS += --disable-compiler-docs
>>> CONFIGURE_OPTIONS += --release-channel=stable
>>> CONFIGURE_OPTIONS += --python=$(PYTHON)
>>> CONFIGURE_OPTIONS += --enable-llvm-link-shared
>>> CONFIGURE_OPTIONS += --llvm-root=$(CLANG_ROOT)
>>>
>>> COMPONENT_PRE_BUILD_ACTION=     rustup default stable;
>>>
>>> # Cleanup standard environment!
>>> COMPONENT_BUILD_ENV =
>>> COMPONENT_BUILD_ENV += CC=$(CC)
>>> COMPONENT_BUILD_ENV += CFLAGS="$(CFLAGS)"
>>> COMPONENT_BUILD_ENV += LDFLAGS="-L$(OPENSSL_PREFIX)/lib/$(MACH64)
>>> -lssl -lcrypto"
>>> COMPONENT_BUILD_ENV += CXX=$(CXX)
>>> COMPONENT_BUILD_ENV += CPPFLAGS="-I$(OPENSSL_PREFIX)/include"
>>> COMPONENT_BUILD_ENV += CXXFLAGS="$(CXXFLAGS)"
>>> COMPONENT_BUILD_ENV += AR=$(GNUAR)
>>> COMPONENT_BUILD_ENV += RUSTC=$(CARGO_HOME)/bin/rustc
>>> # Enforce linker consistency
>>> COMPONENT_BUILD_ENV += RUSTFLAGS="-C linker=$(CC)"
>>> COMPONENT_BUILD_ENV += RUST_BACKTRACE=1
>>>
>>> GMAKE = $(PYTHON)
>>> COMPONENT_BUILD_GMAKE_ARGS =
>>> COMPONENT_BUILD_TARGETS = build
>>> COMPONENT_BUILD_ARGS = ./x.py -vv
>>>
>>> COMPONENT_INSTALL_ARGS = ./x.py
>>> # Cleanup standard environment
>>> COMPONENT_INSTALL_ENV =
>>> COMPONENT_INSTALL_ENV += $(COMPONENT_BUILD_ENV)
>>> # Set install path
>>> COMPONENT_INSTALL_ENV += DESTDIR=$(PROTO_DIR)
>>>
>>> COMPONENT_PRE_INSTALL_ACTION = mkdir -p $(PROTO_DIR)
>>>
>>> RUSTC_BIN_POST = cargo
>>> RUSTC_BIN_POST+= clippy-driver
>>> RUSTC_BIN_POST+= rls
>>> RUSTC_BIN_POST+= rustc
>>> RUSTC_BIN_POST+= rustdoc
>>>
>>> #RUSTC_DRIVER_HASH=637b1017ef7a0c2f
>>> # Create symlinks in place of copies to generate proper link actions
>>> # Edit runpath so that libraries are resolved both in the proto area
>>> and system-wide
>>> #COMPONENT_POST_INSTALL_ACTION= \
>>>    ( $(RM) $(PROTOUSRLIBDIR)/*.so; \
>>>      $(MKDIR) $(PROTOUSRLIBDIR.$(BITS)); \
>>>      $(RM) $(PROTOUSRLIBDIR.$(BITS))/*.so; \
>>>      cd $(PROTOUSRLIBDIR.$(BITS)); \
>>>      $(LN) -s ../rustlib/$(RUST_ARCH)/lib/*.so .; \
>>>      for file in $(RUSTC_BIN_POST); \
>>>      do \
>>>        /usr/bin/elfedit -e \
>>>          'dyn:value -s  RUNPATH
>>> "$(GCC_LIBDIR):$$ORIGIN/../lib/rustlib/$(RUST_ARCH)/lib:$$ORIGIN/../lib/$(MACH64)"'
>>> \
>>>          $(PROTOUSRBINDIR)/$$file ; \
>>>        /usr/bin/elfedit -e \
>>>          'dyn:value -s  RPATH
>>> "$(GCC_LIBDIR):$$ORIGIN/../lib/rustlib/$(RUST_ARCH)/lib:$$ORIGIN/../lib/$(MACH64)"'
>>> \
>>>          $(PROTOUSRBINDIR)/$$file ; \
>>>      done; \
>>>      for file in $(PROTOUSRLIBDIR)/rustlib/$(RUST_ARCH)/lib/*.so; \
>>>      do \
>>>        /usr/bin/elfedit -e 'dyn:value -s  RUNPATH
>>> "$(GCC_LIBDIR):$$ORIGIN:$$ORIGIN/../../$(MACH64)"' $$file ; \
>>>        /usr/bin/elfedit -e 'dyn:value -s  RPATH
>>> "$(GCC_LIBDIR):$$ORIGIN:$$ORIGIN/../../$(MACH64)"' $$file ; \
>>>      done; \
>>>    )
>>>
>>> # Build dependencies (make sure completions are installed)
>>> REQUIRED_PACKAGES += utility/bash-completion
>>> REQUIRED_PACKAGES += $(GCC_RUNTIME_PKG)
>>> REQUIRED_PACKAGES += $(GXX_RUNTIME_PKG)
>>> REQUIRED_PACKAGES += developer/clang-13
>>> REQUIRED_PACKAGES += library/libssh2
>>> REQUIRED_PACKAGES += library/http-parser
>>> # Auto-generated dependencies
>>> REQUIRED_PACKAGES += library/security/openssl-11
>>> REQUIRED_PACKAGES += library/zlib
>>> REQUIRED_PACKAGES += SUNWcs
>>> REQUIRED_PACKAGES += system/library
>>> REQUIRED_PACKAGES += system/library/math
>>> REQUIRED_PACKAGES += web/curl
>>>
>>> Am 18.06.2022 um 19:31 schrieb Gary Mills:
>>>> On Sat, Jun 18, 2022 at 03:49:33PM +0200, Friedrich Kink via oi-dev
>>>> wrote:
>>>>> I try to prepare new rustc package with current version 1.61.0. So
>>>>> far
>>>>> building and installing is already working. But publishing
>>>>> respectively make
>>>>> REQUIRED_PACKAGES immediately bails out with the following error
>>>>> message:
>>>> [...]
>>>>> truss shows that make REQUIRED_PACKAGE really tries to open the file
>>>>> /usr/share/src/myoi-userland/components/developer/rust/RESOLVE_DEPS=
>>>>> Any
>>>>> idea what goes wrong here? To simplify things I just used sample
>>>>> manifest
>>>>> p5m file to exclude home made errors.
>>>> First of all, I'm pleased that somebody else is working on rust: I
>>>> thought I was the only one.
>>>>
>>>> I anticipated that the upgrade would be difficult, but it's essential
>>>> to upgrade any packages that still use clang-90 .  Rust is one of
>>>> these.  Consequently, I did the upgrade by stages.  The first stage
>>>> was to build and publish the original rust package with no
>>>> change.Thanks a
>>>> I found I had to make one change: COMPONENT_PRE_CONFIGURE_ACTION
>>>> had to copy files, instead of creating symlinks or hard links.  Then,
>>>> build and publish were successful
>>>>
>>>> Subsequent stages were to upgrade clang, to upgrade python, and to
>>>> convert the Makefile to the new style.  I'm still stuck in the second
>>>> stage.  You clearly have gotten further.  I found many things that did
>>>> not work, but nothing successful.  The original rust version (1.44.1)
>>>> is too old to build with clang-13.  I chose 1.60.0, since it had a
>>>> bootstrap archive available from Joyent.  I also found that this
>>>> version will not build with the clang compilers.  I had to revert to
>>>> gcc for these.  Still, my builds terminated with this error:
>>>>
>>>>      libLLVM-14-rust-1.60.0-stable.so is missing
>>>>
>>>> I don't know how to get past that error.
>>>>
>>>
>>> _______________________________________________
>>> oi-dev mailing list
>>> oi-dev at openindiana.org
>>> https://openindiana.org/mailman/listinfo/oi-dev
>>
>> _______________________________________________
>> oi-dev mailing list
>> oi-dev at openindiana.org
>> https://openindiana.org/mailman/listinfo/oi-dev
>>
>
> _______________________________________________
> oi-dev mailing list
> oi-dev at openindiana.org
> https://openindiana.org/mailman/listinfo/oi-dev