[rustup] Rust-lang package manager

Grommish · March 2, 2024, 9:39am

There is an existing rust-lang package that I worked on that was completed after I went on sabbatical. I had issues prior to leaving with massive compile times and issues when switching targets. While gone, I decided to look into using rustup instead when I got back. This wasn't an option for me originally as I didn't know enough about either the Openwrt build system or rust-lang in general.

This is what rustup currently solves. Rustup does NOT require a rebuild between targets, since it doesn't actually compile anything locally for rust and will/should handle any target arch supported by rust natively.

Tuple defines can be supported upstream for specific Openwrt targets (mips64-openwrt-linux-musl is already supported natively as a Tier 3 target by rust-lang). It's a matter of testing and submitting to the upstream, which can always come later to help get rid of the nasty kludge logic to figure out the target tuples.

I also, at the moment, install cross though I'm not sure just how useful it will end up being since it seems to require docker implementations and other things. I've not had a call to use it yet, so shrug. I may pull it out before to long.

For anyone who wants to help me test, you can check out the commit at https://github.com/openwrt/packages/compare/master...Grommish:packages:rustup

To use:

Add the following to a Rust-based application package Makefile:

include $(TOPDIR)/package/feeds/packages/rustup/rust-values.mk

PKG_BUILD_DEPENDS:=rustup/host

For applications that use cargo calls, you can use:

define Build/Compile
        cd $(PKG_BUILD_DIR) && \
        $(CARGO_PKG_CONFIG_VARS) \
        $(CARGO) $(CARGO_PKG_CONFIG_ARGS)
endef

as the compile option.

grommish@DESKTOP-AW:~/openwrt$ time make package/feeds/packages/rustup/host/compile
info: downloading installer
warning: downloading with complete profile isn't recommended unless you are a developer of the rust language
verbose: creating home directory: '/home/grommish/openwrt/staging_dir/host/.rustup/'
info: profile set to 'complete'
info: setting default host triple to x86_64-unknown-linux-gnu
verbose: creating update-hash directory: '/home/grommish/openwrt/staging_dir/host/.rustup/update-hashes'
verbose: installing toolchain 'nightly-x86_64-unknown-linux-gnu'
verbose: toolchain directory: '/home/grommish/openwrt/staging_dir/host/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu'
info: syncing channel updates for 'nightly-x86_64-unknown-linux-gnu'
verbose: creating temp root: /home/grommish/openwrt/staging_dir/host/.rustup/tmp
verbose: creating temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/tspbf9h7sv8p7lyn_file
verbose: downloading file from: 'https://static.rust-lang.org/dist/channel-rust-nightly.toml.sha256'
verbose: downloading with reqwest
verbose: deleted temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/tspbf9h7sv8p7lyn_file
verbose: no update hash at: '/home/grommish/openwrt/staging_dir/host/.rustup/update-hashes/nightly-x86_64-unknown-linux-gnu'
verbose: creating temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/mbw5_3zw7kwmprox_file.toml
verbose: downloading file from: 'https://static.rust-lang.org/dist/channel-rust-nightly.toml'
verbose: downloading with reqwest
verbose: checksum passed
verbose: deleted temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/mbw5_3zw7kwmprox_file.toml
info: latest update on 2024-03-02, rust version 1.78.0-nightly (2dceda4f3 2024-03-01)
info: downloading component 'cargo'
verbose: creating Download Directory directory: '/home/grommish/openwrt/staging_dir/host/.rustup/downloads'
verbose: downloading file from: 'https://static.rust-lang.org/dist/2024-03-02/cargo-nightly-x86_64-unknown-linux-gnu.tar.xz'
verbose: downloading with reqwest
verbose: checksum passed
info: downloading component 'clippy'
verbose: downloading file from: 'https://static.rust-lang.org/dist/2024-03-02/clippy-nightly-x86_64-unknown-linux-gnu.tar.xz'
verbose: downloading with reqwest
verbose: checksum passed
info: downloading component 'llvm-tools'
verbose: downloading file from: 'https://static.rust-lang.org/dist/2024-03-02/llvm-tools-nightly-x86_64-unknown-linux-gnu.tar.xz'
verbose: downloading with reqwest
verbose: checksum passed
info: downloading component 'miri'
verbose: downloading file from: 'https://static.rust-lang.org/dist/2024-03-02/miri-nightly-x86_64-unknown-linux-gnu.tar.xz'
verbose: downloading with reqwest
verbose: checksum passed
info: downloading component 'rls'
verbose: downloading file from: 'https://static.rust-lang.org/dist/2024-03-02/rls-nightly-x86_64-unknown-linux-gnu.tar.xz'
verbose: downloading with reqwest
verbose: checksum passed
info: downloading component 'rust-analysis'
verbose: downloading file from: 'https://static.rust-lang.org/dist/2024-03-02/rust-analysis-nightly-x86_64-unknown-linux-gnu.tar.xz'
verbose: downloading with reqwest
verbose: checksum passed
info: downloading component 'rust-analyzer'
verbose: downloading file from: 'https://static.rust-lang.org/dist/2024-03-02/rust-analyzer-nightly-x86_64-unknown-linux-gnu.tar.xz'
verbose: downloading with reqwest
verbose: checksum passed
info: downloading component 'rust-docs'
verbose: downloading file from: 'https://static.rust-lang.org/dist/2024-03-02/rust-docs-nightly-x86_64-unknown-linux-gnu.tar.xz'
verbose: downloading with reqwest
verbose: checksum passed
info: downloading component 'rust-src'
verbose: downloading file from: 'https://static.rust-lang.org/dist/2024-03-02/rust-src-nightly.tar.xz'
verbose: downloading with reqwest
verbose: checksum passed
info: downloading component 'rust-std'
verbose: downloading file from: 'https://static.rust-lang.org/dist/2024-03-02/rust-std-nightly-x86_64-unknown-linux-gnu.tar.xz'
verbose: downloading with reqwest
verbose: checksum passed
info: downloading component 'rustc'
verbose: downloading file from: 'https://static.rust-lang.org/dist/2024-03-02/rustc-nightly-x86_64-unknown-linux-gnu.tar.xz'
verbose: downloading with reqwest
verbose: checksum passed
info: downloading component 'rustc-codegen-cranelift'
verbose: downloading file from: 'https://static.rust-lang.org/dist/2024-03-02/rustc-codegen-cranelift-nightly-x86_64-unknown-linux-gnu.tar.xz'
verbose: downloading with reqwest
verbose: checksum passed
info: downloading component 'rustc-dev'
verbose: downloading file from: 'https://static.rust-lang.org/dist/2024-03-02/rustc-dev-nightly-x86_64-unknown-linux-gnu.tar.xz'
verbose: downloading with reqwest
verbose: checksum passed
info: downloading component 'rustfmt'
verbose: downloading file from: 'https://static.rust-lang.org/dist/2024-03-02/rustfmt-nightly-x86_64-unknown-linux-gnu.tar.xz'
verbose: downloading with reqwest
verbose: checksum passed
info: installing component 'cargo'
verbose: creating temp directory: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/vhbwr22dar8g8t71_dir
verbose: deleted temp directory: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/vhbwr22dar8g8t71_dir
info: installing component 'clippy'
verbose: creating temp directory: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/0k1lz2xfdrfvivno_dir
verbose: creating temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/nzgoays1lqcjv9ae_file
verbose: creating temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/ejny08uv59nhw72d_file
verbose: deleted temp directory: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/0k1lz2xfdrfvivno_dir
info: installing component 'llvm-tools'
verbose: creating temp directory: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/v6gvf9k1dauol17__dir
verbose: creating temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/hyox8woa2gaw5__a_file
verbose: creating temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/6z40qilso2u23hba_file
verbose: deleted temp directory: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/v6gvf9k1dauol17__dir
info: installing component 'miri'
verbose: creating temp directory: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/kyvkj4rg3gl68lq7_dir
verbose: creating temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/g19qc2ztnuz14b20_file
verbose: creating temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/sp87zk89bbyjzqlg_file
verbose: deleted temp directory: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/kyvkj4rg3gl68lq7_dir
info: installing component 'rls'
verbose: creating temp directory: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/9m7877qdx7alg6qd_dir
verbose: creating temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/quaolcjvar4z_2yc_file
verbose: creating temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/tav9p30lvo4230tb_file
verbose: deleted temp directory: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/9m7877qdx7alg6qd_dir
info: installing component 'rust-analysis'
verbose: creating temp directory: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/cc9c6eur9fvqmx6b_dir
verbose: creating temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/4blwl_w9fk9jzw_8_file
verbose: creating temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/b_mvxhfgy34baxy1_file
verbose: deleted temp directory: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/cc9c6eur9fvqmx6b_dir
info: installing component 'rust-analyzer'
verbose: creating temp directory: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/u3hekvkdllni1d51_dir
verbose: creating temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/3r09op14d36tq2mz_file
verbose: creating temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/uae9cxc9fpk7bmiq_file
verbose: deleted temp directory: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/u3hekvkdllni1d51_dir
info: installing component 'rust-docs'
verbose: creating temp directory: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/7afi2cj55jyefrsy_dir
verbose: creating temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/5nnd4cps93pjlaqg_file
verbose: creating temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/tl5m6g9r4lrlji9w_file
verbose: deleted temp directory: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/7afi2cj55jyefrsy_dir
info: installing component 'rust-src'
verbose: creating temp directory: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/zcdy3hiw_gej9hnb_dir
verbose: creating temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/hc118tykwvi7hfbr_file
verbose: creating temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/rtqe9xynrnhop1nd_file
verbose: deleted temp directory: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/zcdy3hiw_gej9hnb_dir
info: installing component 'rust-std'
verbose: creating temp directory: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/dh4jrl3jc1lichaj_dir
verbose: creating temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/yaavjaec7gy6fbny_file
verbose: creating temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/eiwmd__4q1n4wawu_file
verbose: deleted temp directory: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/dh4jrl3jc1lichaj_dir
info: installing component 'rustc'
verbose: creating temp directory: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/uuu0tdwco2py_sa1_dir
verbose: creating temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/ou_e8oizlhyk3_dq_file
verbose: creating temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/9ei6cgv571n5oa0f_file
verbose: deleted temp directory: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/uuu0tdwco2py_sa1_dir
info: installing component 'rustc-codegen-cranelift'
verbose: creating temp directory: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/eaw1m7bpejtk0kzf_dir
verbose: creating temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/d_5bw2aiwbapcif4_file
verbose: creating temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/42kjqjajdgjua3f2_file
verbose: deleted temp directory: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/eaw1m7bpejtk0kzf_dir
info: installing component 'rustc-dev'
verbose: creating temp directory: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/szb40jl_dujialdu_dir
verbose: creating temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/xqd1wjpiovz1nx9f_file
verbose: creating temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/ryc6dvq8pffbbw7x_file
verbose: deleted temp directory: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/szb40jl_dujialdu_dir
info: installing component 'rustfmt'
verbose: creating temp directory: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/8xc688fr7s3ade_e_dir
verbose: creating temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/3ydxkkhcc649_8qf_file
verbose: creating temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/eejq2k8r5_s4jo1p_file
verbose: deleted temp directory: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/8xc688fr7s3ade_e_dir
verbose: deleted temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/nzgoays1lqcjv9ae_file
verbose: deleted temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/ejny08uv59nhw72d_file
verbose: deleted temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/hyox8woa2gaw5__a_file
verbose: deleted temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/6z40qilso2u23hba_file
verbose: deleted temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/g19qc2ztnuz14b20_file
verbose: deleted temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/sp87zk89bbyjzqlg_file
verbose: deleted temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/quaolcjvar4z_2yc_file
verbose: deleted temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/tav9p30lvo4230tb_file
verbose: deleted temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/4blwl_w9fk9jzw_8_file
verbose: deleted temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/b_mvxhfgy34baxy1_file
verbose: deleted temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/3r09op14d36tq2mz_file
verbose: deleted temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/uae9cxc9fpk7bmiq_file
verbose: deleted temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/5nnd4cps93pjlaqg_file
verbose: deleted temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/tl5m6g9r4lrlji9w_file
verbose: deleted temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/hc118tykwvi7hfbr_file
verbose: deleted temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/rtqe9xynrnhop1nd_file
verbose: deleted temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/yaavjaec7gy6fbny_file
verbose: deleted temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/eiwmd__4q1n4wawu_file
verbose: deleted temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/ou_e8oizlhyk3_dq_file
verbose: deleted temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/9ei6cgv571n5oa0f_file
verbose: deleted temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/d_5bw2aiwbapcif4_file
verbose: deleted temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/42kjqjajdgjua3f2_file
verbose: deleted temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/xqd1wjpiovz1nx9f_file
verbose: deleted temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/ryc6dvq8pffbbw7x_file
verbose: deleted temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/3ydxkkhcc649_8qf_file
verbose: deleted temp file: /home/grommish/openwrt/staging_dir/host/.rustup/tmp/eejq2k8r5_s4jo1p_file
verbose: toolchain 'nightly-x86_64-unknown-linux-gnu' installed
info: default toolchain set to 'nightly-x86_64-unknown-linux-gnu'
    Updating crates.io index
  Downloaded cross v0.2.5
  Downloaded 1 crate (228.9 KB) in 1.08s
  Installing cross v0.2.5
    Updating crates.io index
  Downloaded clap_lex v0.2.4
  Downloaded ctrlc v3.4.2
  Downloaded fastrand v2.0.1
  Downloaded eyre v0.6.12
  Downloaded clap v3.2.25
  Downloaded proc-macro2 v1.0.78
  Downloaded libc v0.2.153
  Downloaded gimli v0.28.1
  Downloaded syn v2.0.52
  Downloaded syn v1.0.109
  Downloaded rustix v0.38.31
  Downloaded os_str_bytes v6.6.1
  Downloaded linux-raw-sys v0.4.13
  Downloaded once_cell v1.19.0
  Downloaded nix v0.27.1
  Downloaded nix v0.24.3
  Downloaded memchr v2.7.1
  Downloaded indexmap v1.9.3
  Downloaded home v0.5.9
  Downloaded object v0.32.2
  Downloaded errno v0.3.8
  Downloaded which v4.4.2
  Downloaded version_check v0.9.4
  Downloaded unicode-ident v1.0.12
  Downloaded toml v0.5.11
  Downloaded thiserror-impl v1.0.57
  Downloaded thiserror v1.0.57
  Downloaded textwrap v0.16.1
  Downloaded termcolor v1.4.1
  Downloaded tempfile v3.10.1
  Downloaded supports-color v1.3.1
  Downloaded strsim v0.10.0
  Downloaded shell-words v1.1.0
  Downloaded serde_json v1.0.114
  Downloaded serde_ignored v0.1.10
  Downloaded serde_derive v1.0.197
  Downloaded ryu v1.0.17
  Downloaded rustc_version v0.4.0
  Downloaded proc-macro-error-attr v1.0.4
  Downloaded itoa v1.0.10
  Downloaded is_ci v1.2.0
  Downloaded indenter v0.3.3
  Downloaded heck v0.4.1
  Downloaded hashbrown v0.12.3
  Downloaded either v1.10.0
  Downloaded dirs-sys v0.3.7
  Downloaded directories v4.0.1
  Downloaded const-sha1 v0.2.0
  Downloaded clap_derive v3.2.25
  Downloaded cc v1.0.88
  Downloaded bitflags v2.4.2
  Downloaded bitflags v1.3.2
  Downloaded backtrace v0.3.69
  Downloaded autocfg v1.1.0
  Downloaded atty v0.2.14
  Downloaded adler v1.0.2
  Downloaded addr2line v0.21.0
  Downloaded cfg-if v1.0.0
  Downloaded serde v1.0.197
  Downloaded semver v1.0.22
  Downloaded rustc-demangle v0.1.23
  Downloaded quote v1.0.35
  Downloaded proc-macro-error v1.0.4
  Downloaded owo-colors v3.5.0
  Downloaded miniz_oxide v0.7.2
  Downloaded color-eyre v0.6.2
  Downloaded shell-escape v0.1.5
  Downloaded 67 crates (6.8 MB) in 1.00s (largest was `linux-raw-sys` at 1.5 MB)
   Compiling proc-macro2 v1.0.78
   Compiling unicode-ident v1.0.12
   Compiling libc v0.2.153
   Compiling version_check v0.9.4
   Compiling cfg-if v1.0.0
   Compiling serde v1.0.197
   Compiling bitflags v2.4.2
   Compiling syn v1.0.109
   Compiling rustix v0.38.31
   Compiling cc v1.0.88
   Compiling autocfg v1.1.0
   Compiling proc-macro-error-attr v1.0.4
   Compiling proc-macro-error v1.0.4
   Compiling backtrace v0.3.69
   Compiling indexmap v1.9.3
   Compiling memchr v2.7.1
   Compiling eyre v0.6.12
   Compiling linux-raw-sys v0.4.13
   Compiling is_ci v1.2.0
   Compiling adler v1.0.2
   Compiling once_cell v1.19.0
   Compiling semver v1.0.22
   Compiling gimli v0.28.1
   Compiling miniz_oxide v0.7.2
   Compiling hashbrown v0.12.3
   Compiling serde_json v1.0.114
   Compiling quote v1.0.35
   Compiling heck v0.4.1
   Compiling syn v2.0.52
   Compiling object v0.32.2
   Compiling indenter v0.3.3
   Compiling bitflags v1.3.2
   Compiling os_str_bytes v6.6.1
   Compiling thiserror v1.0.57
   Compiling rustc-demangle v0.1.23
   Compiling clap_lex v0.2.4
   Compiling atty v0.2.14
   Compiling supports-color v1.3.1
   Compiling owo-colors v3.5.0
   Compiling dirs-sys v0.3.7
   Compiling nix v0.27.1
   Compiling addr2line v0.21.0
   Compiling serde_derive v1.0.197
   Compiling thiserror-impl v1.0.57
   Compiling either v1.10.0
   Compiling termcolor v1.4.1
   Compiling textwrap v0.16.1
   Compiling fastrand v2.0.1
   Compiling ryu v1.0.17
   Compiling strsim v0.10.0
   Compiling cross v0.2.5
   Compiling itoa v1.0.10
   Compiling home v0.5.9
   Compiling which v4.4.2
   Compiling tempfile v3.10.1
   Compiling ctrlc v3.4.2
   Compiling rustc_version v0.4.0
   Compiling directories v4.0.1
   Compiling nix v0.24.3
   Compiling shell-escape v0.1.5
   Compiling const-sha1 v0.2.0
   Compiling shell-words v1.1.0
   Compiling clap_derive v3.2.25
   Compiling color-eyre v0.6.2
   Compiling clap v3.2.25
   Compiling serde_ignored v0.1.10
   Compiling toml v0.5.11
    Finished `release` profile [optimized] target(s) in 1m 32s
  Installing /home/grommish/openwrt/staging_dir/host/.cargo/bin/cross
  Installing /home/grommish/openwrt/staging_dir/host/.cargo/bin/cross-util
   Installed package `cross v0.2.5` (executables `cross`, `cross-util`)

real    4m54.741s
user    6m24.162s
sys     0m30.732s

I currently just symlink the executables to the staging_dir/host/bin to make it available across the fakeroot.

ln -s /home/grommish/openwrt/staging_dir/host/.cargo//bin/cargo /home/grommish/openwrt/staging_dir/host/bin/cargo
ln -s /home/grommish/openwrt/staging_dir/host/.cargo//bin/cargo-clippy /home/grommish/openwrt/staging_dir/host/bin/cargo-clippy
ln -s /home/grommish/openwrt/staging_dir/host/.cargo//bin/cargo-fmt /home/grommish/openwrt/staging_dir/host/bin/cargo-fmt
ln -s /home/grommish/openwrt/staging_dir/host/.cargo//bin/cargo-miri /home/grommish/openwrt/staging_dir/host/bin/cargo-miri
ln -s /home/grommish/openwrt/staging_dir/host/.cargo//bin/clippy-driver /home/grommish/openwrt/staging_dir/host/bin/clippy-driver
ln -s /home/grommish/openwrt/staging_dir/host/.cargo//bin/cross /home/grommish/openwrt/staging_dir/host/bin/cross
ln -s /home/grommish/openwrt/staging_dir/host/.cargo//bin/cross-util /home/grommish/openwrt/staging_dir/host/bin/cross-util
ln -s /home/grommish/openwrt/staging_dir/host/.cargo//bin/rls /home/grommish/openwrt/staging_dir/host/bin/rls
ln -s /home/grommish/openwrt/staging_dir/host/.cargo//bin/rust-analyzer /home/grommish/openwrt/staging_dir/host/bin/rust-analyzer
ln -s /home/grommish/openwrt/staging_dir/host/.cargo//bin/rust-gdb /home/grommish/openwrt/staging_dir/host/bin/rust-gdb
ln -s /home/grommish/openwrt/staging_dir/host/.cargo//bin/rust-gdbgui /home/grommish/openwrt/staging_dir/host/bin/rust-gdbgui
ln -s /home/grommish/openwrt/staging_dir/host/.cargo//bin/rust-lldb /home/grommish/openwrt/staging_dir/host/bin/rust-lldb
ln -s /home/grommish/openwrt/staging_dir/host/.cargo//bin/rustc /home/grommish/openwrt/staging_dir/host/bin/rustc
ln -s /home/grommish/openwrt/staging_dir/host/.cargo//bin/rustdoc /home/grommish/openwrt/staging_dir/host/bin/rustdoc
ln -s /home/grommish/openwrt/staging_dir/host/.cargo//bin/rustfmt /home/grommish/openwrt/staging_dir/host/bin/rustfmt
ln -s /home/grommish/openwrt/staging_dir/host/.cargo//bin/rustup /home/grommish/openwrt/staging_dir/host/bin/rustup

brocaar · March 5, 2024, 3:38pm

Hi @Grommish, that is really cool. I have not tried it (yet), but I do have a question. As mips targets are tier 3, these can't be installed using rustup.

When I was looking for a solution, I found:

However, wouldn't that mean that you would always need Rust nightly to build for tier 3 targets? That might be a problem for some... What do you think?

Btw, I don't think you need cross. The purpose of cross is to setup a cross-compile environment, e.g. to link against libraries for the target platform. The OpenWrt build system already provides this.

Grommish · March 5, 2024, 11:23pm

I don't need cross, correct. However, Tier 3 targets ARE supported natively, even by Rustup, although I never bothered to build out std for the Tier 3 target because we didn't need it.. We only need the definitions for the toolchain. Realistically, I doubt anyone would be using the rust compiler ON the device (as in, compiling a rust application from a router/switch), so it doesn't need to be anything other than Tier 3.

mips64-openwrt-linux-musl is an official Tier 3 target, so it's recognized by cargo/rustc which is all it needs.. It can cross for the target arch with those definitions.

Eventually, with enough testing, I can get specific Openwrt targets upstreamed as Tier 3 to rust and get rid of all the nasty klanged target logic that has to be done, plus allow specific targets to use FPU/VFP natively in the compiled applications rather than being generic.

Currently, Openwrt calls ALL ARM devices arm-unknown-linux-xxxx which doesn't allow for optimizations for armv7 or aarch64 or armv5, etc. That can be fixed by outlining and upstreaming the correct Openwrt tuples.

As for 'rebuilding nightly', no.. The nightly toolchain is used so -Z build-std can be called, but it installs the toolchain that is available at the time whoever builds out. The toolchain itself doesn't get rebuilt unless a make dirclean is called, or a make package/feeds/packages/rustup/host/clean is called because as far as the build system is concerned, the rustup install doesn't change. If something NEEDS to be force-reinstalled (say, a security release) then a bump to the rustup package Makefile will force a "rebuild", an uninstall and redownload of the new "nightly" toolchain.,

"Compile" and setup time for rustup, on my machine, is about 5 minutes real-time before it's ready to go, including installing cross. Building rust-lang on the same hardware before I left took 3+ hours and had to be done for EACH target change.. Which isn't what should have happened, given the fact it's a build host tool living in staging_dir and not target dependant.

brocaar · March 6, 2024, 11:58am

I don't need cross, correct. However, Tier 3 targets ARE supported natively, even by Rustup

What I mean is that rustup doesn't provide pre-compiled binaries for tier 3 targets.

rustup target add mips-unknown-linux-musl
error: component 'rust-std' for target 'mips-unknown-linux-musl' is unavailable for download for channel '1.76.0'

From what I understood, you need the nightly version to cross-compile to these targets, e.g.:

rustup +nightly component add rust-src
cargo +nightly build -Z build-std --target mips-unknown-linux-musl

As for 'rebuilding nightly', no.. The nightly toolchain is used so -Z build-std can be called, but it installs the toolchain that is available at the time whoever builds out.

But couldn't that be a problem over time? I do understand that the toolchain isn't re-installed on each package compile only during the first build, but doesn't this mean that if two users compile the same package at two different times, they might be using different toolchain versions because it will download what is at that moment in time the "nightly" version? Thus we can't guarantee the toolchain version that will be used. That will become problematic over time, because code that compiles with the current nightly might not compile with a future nightly version. As well, for production systems, using nightly might not be an acceptable choice because it is less tested.

I think it is great when we can reduce the Rust build time from hours to minutes, but I think it is important to make sure that code that compiles today, will still compile tomorrow, next year and the years after.

brocaar · March 8, 2024, 1:03pm

This is not an OpenWrt build, but I think it is relevant to this discussion about using nightly. I just encountered a build failure when compiling one of my projects (https://github.com/chirpstack/chirpstack-gateway-relay/blob/master/Makefile#L24) that uses +nightly to cross-compile to MIPS. The only difference is the nightly version that is used:

Works fine: 1.78.0-nightly (2bf78d12d 2024-02-18)
Fails: 1.78.0-nightly (9c3ad802d 2024-03-07)

(I re-tested this, for both builds I did a cargo clean before the build)

error: linking with `mips-linux-muslsf-gcc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/rust/lib/rustlib/x86_64-unknown-linux-gnu/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/rust/bin" VSLANG="1033" "mips-linux-muslsf-gcc" "/tmp/rustcgir3dT/symbols.o" "/target/mips-unknown-linux-musl/release/deps/chirpstack_gateway_relay-0e78e2e912136be8.chirpstack_gateway_relay.e76cb6136f7bb532-cgu.0.rcgu.o" "-Wl,--as-needed" "-L" "/target/mips-unknown-linux-musl/release/deps" "-L" "/target/release/deps" "-L" "/rust/lib/rustlib/mips-unknown-linux-musl/lib" "-Wl,-Bstatic" "/target/mips-unknown-linux-musl/release/deps/libcompiler_builtins-ab1267840da40625.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/rust/lib/rustlib/mips-unknown-linux-musl/lib" "-o" "/target/mips-unknown-linux-musl/release/deps/chirpstack_gateway_relay-0e78e2e912136be8" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-Wl,--strip-all" "-nodefaultlibs"
  = note: /usr/local/bin/../lib/gcc/mips-linux-muslsf/9.2.0/../../../../mips-linux-muslsf/bin/ld: /target/mips-unknown-linux-musl/release/deps/chirpstack_gateway_relay-0e78e2e912136be8.chirpstack_gateway_relay.e76cb6136f7bb532-cgu.0.rcgu.o: in function `std::sys::pal::unix::thread::Thread::get_name':
          chirpstack_gateway_relay.e76cb6136f7bb532-cgu.0:(.text._ZN3std3sys3pal4unix6thread6Thread8get_name17h9409b87903ef8b26E+0x58): undefined reference to `pthread_getname_np'
          /usr/local/bin/../lib/gcc/mips-linux-muslsf/9.2.0/../../../../mips-linux-muslsf/bin/ld: chirpstack_gateway_relay.e76cb6136f7bb532-cgu.0:(.text._ZN3std3sys3pal4unix6thread6Thread8get_name17h9409b87903ef8b26E+0x64): undefined reference to `pthread_getname_np'
          collect2: error: ld returned 1 exit status
          
  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#rustc-link-lib)

I think this shows that using +nightly can be tricky, especially when the used nightly version depends on when the user will compile the Rust package for the first time.

Grommish · March 8, 2024, 1:42pm

Well, that's one way to look at it.. However:

undefined reference to `pthread_getname_np'

Since the current version of rust and/or rustup uses x86_64-gcc as the base to cross compile, it could very well be an issue with the chirpstack project not being made for whatever target you are trying to cross compile to.

When I originally went looking for test applications to package, someone suggested a package (because they wanted to use it, of course) that had some encryption library in it.. That encryption library was x86_64 only. It was never designed to work off that arch.

Now, obviously that isn't what you re seeing, but it's more plausible that the application code changed versus the rust-lang project removing a reference in a library, or crate. Or, the reference was moved crates and the project you are building just hasn't updated.

The entire rust language itself is a moving target with a 6 week build/release schedule for the official repo, not counting nightlies.

That being said, I can lock down a specific date for the nightlies and force everyone to just use that, if I can't come up with a different, more eligant solution.

brocaar · March 11, 2024, 11:03am

That is not the issue, This is running fine on MIPS targets. Really the only difference was the difference between the Rust nightly version. I have reproduced this issue again for you:

rustup toolchain add nightly-2024-02-18-x86_64-unknown-linux-gnu
rustup toolchain add nightly-2024-03-10-x86_64-unknown-linux-gnu
cargo clean
cross +nightly-2024-02-18 build -Z build-std=panic_abort,std --target mips-unknown-linux-musl --release
- Compiles fine
cargo clean
cross +nightly-2024-03-10 build -Z build-std=panic_abort,std --target mips-unknown-linux-musl --release
- Results in the above compile error

In summary I think:

Builds should always be reproducible
Using Rust nightly might not always be an acceptable choice for production systems

I as well would really like to avoid spending several hours on building the Rust toolchain from source, but I'm not sure if using rustup without using nightly but for tier 3 targets is possible. For the MIPS binaries that I build using cross-rs (thus not using the OpenWrt build environment), I still need to figure out how I want to move forward with this.

Grommish · March 12, 2024, 12:45am

The biggest issue is that there is zero point in making/promoting the xxx-openwrt-linux-musl toolchains to being Tier 2 targets. I was thinking about perhaps a hybrid approach where the x86_64 (or darwin) stuff can be gotten by rustup and then the Stage1/2 artifacts (ie: libstd) for the target can be made on device.. Once the libstd is created, dropping it into staging_dir/libs makes it a "valid" target.

As I just started a new job, it isn't the highest priority for me, though I continue to work on it.. I've been talking with the rust devs and going on things, the needs, wants, restrictions,etc.

predators_46 · March 27, 2024, 1:44pm

Thank you for making this rustup package very useful for those who need +nightly

predators_46 · March 27, 2024, 1:46pm

use musl version 1.2.4 fix pthread_getname_np