Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Intermittent "no entry found for key" error in infrastructure/testdriver tests #23964

Open
georgeroman opened this issue Aug 13, 2019 · 7 comments
Labels
A-webrender I-panic Servo encounters a panic. I-race A race between two concurrent pieces of code.

Comments

@georgeroman
Copy link
Contributor

georgeroman commented Aug 13, 2019

no entry found for key (thread WRRenderBackend#0, at src/libcore/option.rs:1166) appears most of the time when running the infrastructure/testdriver tests. I haven't been able to reproduce the error outside the test environment, but instead manage to tweak the test runner to use a debugging version of Servo in order to get access to the stacktrace.

Stacktrace
Thread 13 "WRRenderBackend" received signal SIGSEGV, Segmentation fault.  
[Switching to Thread 0x7ffff03ff700 (LWP 6333)]
0x00007ffff6e50c87 in __strlen_avx2 () from /usr/lib/libc.so.6
(gdb) where
#0  0x00007ffff6e50c87 in __strlen_avx2 () at /usr/lib/libc.so.6
#1  0x000055555ed048d8 in backtrace::symbolize::libbacktrace::Symbol::name::{{closure}}     (ptr=0x7fff63e41325 <error: Cannot access memory at address 0x7fff63e41325>)
at /home/george/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.26/src/symbolize/libbacktrace.rs:68
#2  0x000055555ed04789 in backtrace::symbolize::libbacktrace::Symbol::name (self=0x7ffff03e82f0) at /home/george/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.26/src/symbolize/libbacktrace.rs:77
#3  0x000055555ed0188e in backtrace::symbolize::Symbol::name (self=0x7ffff03e82f0) at /home/george/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.26/src/symbolize/mod.rs:179
#4  0x000055555ecf4f54 in backtrace::capture::Backtrace::resolve::{{closure}} (symbol=0x7ffff03e82f0) at /home/george/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.26/src/capture.rs:212
#5  0x000055555ed04c9a in backtrace::symbolize::libbacktrace::syminfo_cb (data=0x7ffff03e8460, pc=93825151295980, symname=0x7fff63e41325 <error: Cannot access memory at address 0x7fff63e41325>, _symval=93825151295936, _symsize=52)
at /home/george/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.26/src/symbolize/libbacktrace.rs:194
#6  0x000055555ed0971b in elf_syminfo
(state=0x7fffbc001060, addr=93825151295980, callback=0x55555ed04b80 <backtrace::symbolize::libbacktrace::syminfo_cb>, error_callback=0x55555ed04b60 <backtrace::symbolize::libbacktrace::error_cb>, data=0x7ffff03e8460)
at src/libbacktrace/elf.c:760
#7  0x000055555ed0739a in __rbt_backtrace_syminfo
(state=0x7fffbc001060, pc=93825151295980, callback=0x55555ed04b80 <backtrace::symbolize::libbacktrace::syminfo_cb>, error_callback=0x55555ed04b60 <backtrace::symbolize::libbacktrace::error_cb>, data=0x7ffff03e8460)
at src/libbacktrace/fileline.c:199
#8  0x000055555ed04fd2 in backtrace::symbolize::libbacktrace::resolve (what=..., cb=...) at /home/george/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.26/src/symbolize/libbacktrace.rs:441
#9  0x000055555ed01840 in backtrace::symbolize::resolve_frame_unsynchronized (frame=0x7fffc3e26e08, cb=...) at /home/george/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.26/src/symbolize/mod.rs:152
#10 0x000055555ed016d0 in backtrace::symbolize::resolve_frame (frame=0x7fffc3e26e08, cb=...) at /home/george/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.26/src/symbolize/mod.rs:105
#11 0x000055555ecf4ce4 in backtrace::capture::Backtrace::resolve (self=0x7ffff03e8648) at /home/george/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.26/src/capture.rs:219
#12 0x000055555ecf4862 in backtrace::capture::Backtrace::new () at /home/george/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.26/src/capture.rs:127
#13 0x000055555630a4af in servo::main::{{closure}} (info=0x7ffff03e8b68) at ports/glutin/main2.rs:116
#14 0x000055555f543b26 in std::panicking::rust_panic_with_hook () at src/libstd/panicking.rs:481
#15 0x000055555f5435c2 in std::panicking::continue_panic_fmt () at src/libstd/panicking.rs:384
#16 0x000055555f5434a6 in rust_begin_unwind () at src/libstd/panicking.rs:311
#17 0x000055555f56624d in core::panicking::panic_fmt () at src/libcore/panicking.rs:85
#18 0x000055555f5662b7 in core::option::expect_failed () at src/libcore/option.rs:1166
#19 0x000055555d67b5ea in core::option::Option<T>::expect (self=..., msg=...) at /rustc/dddb7fca09dc817ba275602b950bb81a9032fb6d/src/libcore/option.rs:345
#20 0x000055555d7f8a9d in <std::collections::hash::map::HashMap<K,V,S> as core::ops::index::Index<&Q>>::index (self=0x7fffc3e26330, key=0x7ffff03e8da0)
at /rustc/dddb7fca09dc817ba275602b950bb81a9032fb6d/src/libstd/collections/hash/map.rs:1041
#21 0x000055555d6c1bab in webrender::hit_test::HitTester::get_pipeline_root (self=0x7fffc3e262f8, pipeline_id=...) at /home/george/.cargo/git/checkouts/webrender-c3596abe1cf4f320/9e35bec/webrender/src/hit_test.rs:509
#22 0x000055555d6c1f90 in webrender::hit_test::HitTest::get_absolute_point::{{closure}} (id=...) at /home/george/.cargo/git/checkouts/webrender-c3596abe1cf4f320/9e35bec/webrender/src/hit_test.rs:568
#23 0x000055555d688b7b in core::option::Option<T>::and_then (self=..., f=...) at /rustc/dddb7fca09dc817ba275602b950bb81a9032fb6d/src/libcore/option.rs:655
#24 0x000055555d6c1f1a in webrender::hit_test::HitTest::get_absolute_point (self=0x7ffff03e9b48, hit_tester=0x7fffc3e262f8) at /home/george/.cargo/git/checkouts/webrender-c3596abe1cf4f320/9e35bec/webrender/src/hit_test.rs:566
#25 0x000055555d6c085a in webrender::hit_test::HitTester::hit_test (self=0x7fffc3e262f8, test=...) at /home/george/.cargo/git/checkouts/webrender-c3596abe1cf4f320/9e35bec/webrender/src/hit_test.rs:416
#26 0x000055555d55a329 in webrender::render_backend::Document::process_frame_msg (self=0x7fffc3e26020, message=...) at /home/george/.cargo/git/checkouts/webrender-c3596abe1cf4f320/9e35bec/webrender/src/render_backend.rs:461
#27 0x000055555d5643a3 in webrender::render_backend::RenderBackend::update_document
(self=0x7ffff03f85a8, document_id=..., resource_updates=Vec<webrender_api::api::ResourceUpdate>(len: 0, cap: 0), interner_updates=..., frame_ops=Vec<webrender_api::api::FrameMsg>(len: 1, cap: 1) = {...}, notifications=Vec<webrender_api::api::NotificationRequest>(len: 0, cap: 0), render_frame=false, invalidate_rendered_frame=false, frame_counter=0x7ffff03f685c, profile_counters=0x7ffff03f9350, has_built_scene=false)
at /home/george/.cargo/git/checkouts/webrender-c3596abe1cf4f320/9e35bec/webrender/src/render_backend.rs:1428
#28 0x000055555d56213b in webrender::render_backend::RenderBackend::prepare_transactions
(self=0x7ffff03f85a8, document_ids=Vec<webrender_api::api::DocumentId>(len: 1, cap: 1) = {...}, transaction_msgs=Vec<webrender_api::api::TransactionMsg>(len: 0, cap: 1), frame_counter=0x7ffff03f685c, profile_counters=0x7ffff03f9350)
at /home/george/.cargo/git/checkouts/webrender-c3596abe1cf4f320/9e35bec/webrender/src/render_backend.rs:1327
#29 0x000055555d560acf in webrender::render_backend::RenderBackend::process_api_msg (self=0x7ffff03f85a8, msg=..., profile_counters=0x7ffff03f9350, frame_counter=0x7ffff03f685c)
at /home/george/.cargo/git/checkouts/webrender-c3596abe1cf4f320/9e35bec/webrender/src/render_backend.rs:1219
#30 0x000055555d55e07a in webrender::render_backend::RenderBackend::run (self=0x7ffff03f85a8, profile_counters=...) at /home/george/.cargo/git/checkouts/webrender-c3596abe1cf4f320/9e35bec/webrender/src/render_backend.rs:970
#31 0x000055555daa5043 in webrender::renderer::Renderer::new::{{closure}} () at /home/george/.cargo/git/checkouts/webrender-c3596abe1cf4f320/9e35bec/webrender/src/renderer.rs:2198
#32 0x000055555d498745 in std::sys_common::backtrace::__rust_begin_short_backtrace (f=...) at /rustc/dddb7fca09dc817ba275602b950bb81a9032fb6d/src/libstd/sys_common/backtrace.rs:77
#33 0x000055555d6fb6f4 in std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}} () at /rustc/dddb7fca09dc817ba275602b950bb81a9032fb6d/src/libstd/thread/mod.rs:470
#34 0x000055555daf53a4 in <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once (self=..., _args=()) at /rustc/dddb7fca09dc817ba275602b950bb81a9032fb6d/src/libstd/panic.rs:315
#35 0x000055555d98f92e in std::panicking::try::do_call (data=0x7ffff03fb2c0 "\340t\267\365\377\177\000") at /rustc/dddb7fca09dc817ba275602b950bb81a9032fb6d/src/libstd/panicking.rs:296
#36 0x000055555f54d7ca in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:80
#37 0x000055555d98ea70 in std::panicking::try (f=...) at /rustc/dddb7fca09dc817ba275602b950bb81a9032fb6d/src/libstd/panicking.rs:275
#38 0x000055555db07ef6 in std::panic::catch_unwind (f=...) at /rustc/dddb7fca09dc817ba275602b950bb81a9032fb6d/src/libstd/panic.rs:394
#39 0x000055555d6fb12d in std::thread::Builder::spawn_unchecked::{{closure}} () at /rustc/dddb7fca09dc817ba275602b950bb81a9032fb6d/src/libstd/thread/mod.rs:469
#40 0x000055555dd663d4 in core::ops::function::FnOnce::call_once{{vtable-shim}} () at /rustc/dddb7fca09dc817ba275602b950bb81a9032fb6d/src/libcore/ops/function.rs:235
#41 0x000055555f53219f in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/dddb7fca09dc817ba275602b950bb81a9032fb6d/src/liballoc/boxed.rs:770
#42 0x000055555f54caf0 in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/dddb7fca09dc817ba275602b950bb81a9032fb6d/src/liballoc/boxed.rs:770
#43 0x000055555f54caf0 in std::sys_common::thread::start_thread () at src/libstd/sys_common/thread.rs:13
#44 0x000055555f54caf0 in std::sys::unix::thread::Thread::new::thread_start () at src/libstd/sys/unix/thread.rs:79
#45 0x00007ffff732a57f in start_thread () at /usr/lib/libpthread.so.0
#46 0x00007ffff6dee0e3 in clone () at /usr/lib/libc.so.6

This is where the panick happens. After some more debugging I found the contents of self.spatial_nodes, self.pipeline_root_nodes and pipeline_id:

self.spatial_nodes
Vec<webrender::hit_test::HitTestSpatialNode>(len: 2, cap: 2) = {webrender::hit_test::HitTestSpatialNode {
  pipeline_id: webrender_api::api::PipelineId (
    1,
    1
  ),
    ...
  },
webrender::hit_test::HitTestSpatialNode {
  pipeline_id: webrender_api::api::PipelineId (
    1,
    1
  ),
  ...
  }
}
pipeline_root_nodes
std::collections::hash::map::HashMap<webrender_api::api::PipelineId, webrender::clip_scroll_tree::SpatialNodeIndex, core::hash::BuildHasherDefault<fxhash::FxHasher>> {
  base: hashbrown::map::HashMap<webrender_api::api::PipelineId, webrender::clip_scroll_tree::SpatialNodeIndex, core::hash::BuildHasherDefault<fxhash::FxHasher>> {
  hash_builder: core::hash::BuildHasherDefault<fxhash::FxHasher> (
    core::marker::PhantomData<fxhash::FxHasher>
  ),
  table: hashbrown::raw::RawTable<(webrender_api::api::PipelineId, webrender::clip_scroll_tree::SpatialNodeIndex)> {
    bucket_mask: 1,
    ctrl: core::ptr::non_null::NonNull<u8> {
      pointer: 0x7ffff043a060 "\377/", '\377' <repeats 15 times>, "/\000"
    },
    data: core::ptr::non_null::NonNull<(webrender_api::api::PipelineId, webrender::clip_scroll_tree::SpatialNodeIndex)> {
      pointer: 0x7ffff043a074
    },
    growth_left: 0,
    items: 1,
    marker: core::marker::PhantomData<(webrender_api::api::PipelineId, webrender::clip_scroll_tree::SpatialNodeIndex)>
  }
}

}

pipeline_id
webrender_api::api::PipelineId (
  1,
  2
)

This should be easily reproducible:
./mach test-wpt --product=servodriver tests/wpt/web-platform-tests/infrastructure/testdriver/bless.html

@jdm
Copy link
Member

jdm commented Aug 15, 2019

Looks like you're seeing the same backtrace segfault that @mmatyas reported in #23966 (comment). As for the actual panic, we've seen this before in #21480.

@jdm
Copy link
Member

jdm commented Sep 19, 2019

I can reproduce this panic with ./mach test-wpt --servodriver tests/wpt/web-platform-tests/infrastructure/testdriver/click.html.

@jdm
Copy link
Member

jdm commented Sep 19, 2019

This happens for these tests when a webdriver operation that performs a hit test occurs before the first layout operation occurs. This layout operation is what populates the fields that the hit test relies upon, and when the testdriver code executes immediately during the initial page parse, layout can be suppressed as a performance optimization so the hit test has no information to rely upon.

@jdm
Copy link
Member

jdm commented Sep 19, 2019

To demonstrate this, add button.getBoundingClientRect(); to click.html right before the testdriver operation. This forces a layout to occur before testdriver tries to initiate a hit test.

@jdm
Copy link
Member

jdm commented Sep 19, 2019

Similarly, I can reproduce this if I refresh a page and click really fast as it starts loading.

@jdm
Copy link
Member

jdm commented Sep 19, 2019

Interestingly, I think that this may be distinct from #21480, since APIs like Document::ElementsFromPoint should be triggering a layout operation before the hit test query occurs. I am unable to reproduce this panic with:

<button>hi there</button>
<script>console.log(document.elementFromPoint(30, 30))</script>

@jdm
Copy link
Member

jdm commented Sep 19, 2019

diff --git a/components/layout_thread/lib.rs b/components/layout_thread/lib.rs
index f03ca9f9d0..0289175af6 100644
--- a/components/layout_thread/lib.rs
+++ b/components/layout_thread/lib.rs
@@ -561,6 +561,20 @@ impl LayoutThread {
         let font_cache_receiver =
             ROUTER.route_ipc_receiver_to_new_crossbeam_receiver(ipc_font_cache_receiver);

+        // Let webrender know about this pipeline by sending an empty display list.
+        let mut epoch = Epoch(0);
+        let webrender_api = webrender_api_sender.create_api();
+        let mut txn = webrender_api::Transaction::new();
+        txn.set_display_list(
+            webrender_api::Epoch(epoch.0),
+            None,
+            Default::default(),
+            (id.to_webrender(), Default::default(), Default::default()),
+            false,
+        );
+        webrender_api.send_transaction(webrender_document, txn);
+        epoch.next();
+
         LayoutThread {
             id: id,
             top_level_browsing_context_id: top_level_browsing_context_id,
@@ -588,9 +602,9 @@ impl LayoutThread {
             document_shared_lock: None,
             running_animations: ServoArc::new(RwLock::new(Default::default())),
             expired_animations: ServoArc::new(RwLock::new(Default::default())),
-            epoch: Cell::new(Epoch(0)),
+            epoch: Cell::new(epoch),
             viewport_size: Size2D::new(Au(0), Au(0)),
-            webrender_api: webrender_api_sender.create_api(),
+            webrender_api,
             webrender_document,
             stylist: Stylist::new(device, QuirksMode::NoQuirks),
             rw_data: Arc::new(Mutex::new(LayoutThreadData {

This patch makes the panic disappear, but I'm pretty sure it just hides the symptom - any click event will still be relying on a nonexistent display list so it won't end up clicking anything. I think the right solution is to force an up-to-date display list to be generated whenever an event occurs that requires interacting with a display list.

@jdm jdm added the I-race A race between two concurrent pieces of code. label Sep 20, 2019
@jdm jdm mentioned this issue Oct 11, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-webrender I-panic Servo encounters a panic. I-race A race between two concurrent pieces of code.
Projects
None yet
Development

No branches or pull requests

2 participants