Build error with proc_macro crates and cross-compilation #37958

Closed
nox opened this Issue Nov 23, 2016 · 13 comments

Projects

None yet

4 participants

@nox
Contributor
nox commented Nov 23, 2016 edited

In servo/servo#14292, cross compilation fails with the following log: https://gist.github.com/jdm/ba11e5fe48fcbc186aecb15a857b54a6

xml5ever is a crate that depends on html5ever-atoms, which itself depends on heapsize_derive if the heap_size feature is selected. heapsize_derive defines #[derive(HeapSizeOf)] which xml5ever makes no use of.

@nox
Contributor
nox commented Nov 23, 2016
@alexcrichton
Member

This may have been fixed by #37846, unsure. @jseyfried would know for sure

@nox
Contributor
nox commented Nov 23, 2016

This landed 5 days ago so I doubt it. Will soon retry the branch with today's nightly.

@nox
Contributor
nox commented Nov 23, 2016 edited
@jseyfried jseyfried self-assigned this Nov 23, 2016
@jseyfried
Contributor
jseyfried commented Nov 24, 2016 edited

Now that ordinary crates can reexport proc macros, rustc needs to load extern crates' proc-macro dependencies. During cross compilation, proc-macro dependencies are in a separate directory from ordinary dependencies, but cargo only links the ordinary dependencies.

@jseyfried jseyfried assigned alexcrichton and unassigned jseyfried Nov 24, 2016
@nox
Contributor
nox commented Nov 24, 2016

Could it just be this typo?

@nox
Contributor
nox commented Nov 24, 2016

@jseyfried Did the build directory of proc-macro dependencies change after the 2016-11-02 nightly? Servo already uses proc-macro on that nightly and we have no cross-compilation build errors.

@jseyfried
Contributor
jseyfried commented Nov 24, 2016 edited

@nox

Could it just be this typo?

I don't think that's related but it would be a good idea to fix regardless.

Did the build directory of proc-macro dependencies change after the 2016-11-02 nightly? Servo already uses proc-macro on that nightly and we have no cross-compilation build errors.

No, but the 2016-11-02 nightly doesn't support proc macro reexports, so rustc didn't need to load extern crates' proc-macro dependencies when compiling a crate. Rustc was able to load a crate's own proc-macro dependencies because cargo passes these dependencies directly with --extern proc_macro_crate=path/to/crate.

@alexcrichton said this should be fairly simple to fix on cargo's end.

@emk
Contributor
emk commented Nov 26, 2016 edited

At Faraday, we're trying to deploy our first Rust-based service into production, and it's blocking on this issue. (We cross-compile from Ubuntu to Docker containers running Alpine Linux, and we need proc_macro because our BigML Rust client bindings need to mix serde and macro_rules to avoid code duplication.)

Here's an example. In Cargo.toml:

[package]
name = "cross_proc_macro_example"
version = "0.1.0"

[dependencies]
serde = "0.8"
serde_derive = "0.8"

In src/lib.rs:

#![feature(proc_macro)]

#[macro_use]
extern crate serde_derive;

#[derive(Serialize)]
pub struct Point {
    x: f64,
    y: f64,
}

In examples/example.rs:

extern crate cross_proc_macro_example;

fn main() {}

This works fine:

cargo build --verbose --example=example

This fails:

cargo build --verbose --example=example --target=x86_64-unknown-linux-musl

...with the errors:

     Running `rustc src/lib.rs --crate-name cross_proc_macro_example --crate-type lib -g -C metadata=e2ae6c3617222432 -C extra-filename=-e2ae6c3617222432 --out-dir /home/emk/w/src/pr/cross_proc_macro_example/target/x86_64-unknown-linux-musl/debug/deps --emit=dep-info,link --target x86_64-unknown-linux-musl -L dependency=/home/emk/w/src/pr/cross_proc_macro_example/target/x86_64-unknown-linux-musl/debug/deps --extern serde=/home/emk/w/src/pr/cross_proc_macro_example/target/x86_64-unknown-linux-musl/debug/deps/libserde-05bbf4d3437164cb.rlib --extern serde_derive=/home/emk/w/src/pr/cross_proc_macro_example/target/debug/deps/libserde_derive-4c72d69e014dae87.so`
     Running `rustc examples/example.rs --crate-name example --crate-type bin -g -C metadata=e2ae6c3617222432 -C extra-filename=-e2ae6c3617222432 --out-dir /home/emk/w/src/pr/cross_proc_macro_example/target/x86_64-unknown-linux-musl/debug/examples --emit=dep-info,link --target x86_64-unknown-linux-musl -L dependency=/home/emk/w/src/pr/cross_proc_macro_example/target/x86_64-unknown-linux-musl/debug/deps --extern serde=/home/emk/w/src/pr/cross_proc_macro_example/target/x86_64-unknown-linux-musl/debug/deps/libserde-05bbf4d3437164cb.rlib --extern serde_derive=/home/emk/w/src/pr/cross_proc_macro_example/target/debug/deps/libserde_derive-4c72d69e014dae87.so --extern cross_proc_macro_example=/home/emk/w/src/pr/cross_proc_macro_example/target/x86_64-unknown-linux-musl/debug/deps/libcross_proc_macro_example-e2ae6c3617222432.rlib`
error[E0463]: can't find crate for `serde_derive` which `cross_proc_macro_example` depends on
 --> examples/example.rs:1:1
  |
1 | extern crate cross_proc_macro_example;
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate

error: aborting due to previous error

error: Could not compile `cross_proc_macro_example`.

Are there any workarounds for this issue that might get us unstuck? If not, we could modify our build system to deploy on an Ubuntu container instead of cross-compiling to Alpine.

@emk emk referenced this issue Nov 26, 2016
Closed

Tracking issue for "Macros 1.1" (RFC #1681) #35900

50 of 53 tasks complete
@jseyfried jseyfried assigned jseyfried and unassigned alexcrichton Nov 27, 2016
@jseyfried
Contributor
jseyfried commented Nov 27, 2016 edited

I fixed this in #38024 when no proc macros are reexported (this includes both @nox's and @emk's use cases). The fix will probably be in the 2016-11-28 nightly.

This is still an issue (opened rust-lang/cargo#3334) when proc macros are reexported, either via pub extern crate proc_macros;, #![feature(macro_reexport)], or #![feature(use_extern_macros)].

@jseyfried
Contributor
jseyfried commented Nov 27, 2016 edited

As we wait for #38024 to land, this can be worked around by setting RUSTFLAGS="-L dependency=/path/to/proj/target/(debug|release)/deps".

@bors bors added a commit that referenced this issue Nov 28, 2016
@bors bors Auto merge of #38024 - jseyfried:avoid_needless_proc_macro_deps, r=nrc
Avoid loading needless proc-macro dependencies

Fixes #37958 when no proc-macros are exported; in particular, without `pub extern crate proc_macros;`, `#![feature(macro_reexport)]`, or `#![feature(use_extern_macros)]`.

I opened rust-lang/cargo#3334 for exported proc macros.

r? @alexcrichton
39c267a
@emk
Contributor
emk commented Nov 28, 2016

@jseyfried Thank you for looking into this! I can confirm that your workaround works. I get a clean build using:

env RUSTFLAGS="-L dependency=target/debug/deps" cargo build --verbose --example=example --target=x86_64-unknown-linux-musl
@alexcrichton
Member

Cargo fix submitted as rust-lang/cargo#3335

@bors bors closed this in #38024 Nov 28, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment