Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crash during ascent after dropping boosters #2716

Closed
lpgagnon opened this issue Sep 10, 2020 · 4 comments · Fixed by #2753
Closed

Crash during ascent after dropping boosters #2716

lpgagnon opened this issue Sep 10, 2020 · 4 comments · Fixed by #2753
Labels
Milestone

Comments

@lpgagnon
Copy link

lpgagnon commented Sep 10, 2020

Seen twice; not reproducible at-will, so no journal. Both in Gallai.

From FATAL.log:
F0909 21:55:24.457835 9920 not_null_body.hpp:33] Check failed: other != nullptr

From INFO.log, crash happens both times after a bunch of "removing vessel"/"destroying part" messages

In both cases the scenario was the same: launching a rocket consisting of 2 stages plus a pair of solid boosters, the boosters were dropped a few seconds ago. Some parts have been blowing up, presumably from aerodynamic stress. It's possible enough time elapsed for the dropped parts to fall out of physics range.

Each solid booster (ROE-Algol1) has a pair of separation motors attached (ROSmallSpinMotor).
One difference in the second instance: one of the Algols blew up (testlite) shortly before running out of fuel. Its spinmotors would have become debris a few seconds before staging the remaining Algol.

INFO.log for first instance
INFO.log for second instance

@lpgagnon
Copy link
Author

Journal! new scenario, but same failed check and similar conditions. Crash occurs during crewed reentry, while the decoupled service module is slowly dropping back and bits of it are exploding. Bits are somewhere around 4km distant.

Happened 3 times in a row, got a journal for 3rd instance.
https://drive.google.com/file/d/10ITznV2oS4RnIjG4aic6qJcmMMCQgFnP/view?usp=sharing

@lpgagnon
Copy link
Author

supporting evidence that it has something to do with the decoupled part: reloaded, deleted it via tracking station, went back to capsule, no crash

Sorry, something went wrong.

@pleroy
Copy link
Member

pleroy commented Oct 7, 2020

Decode stack trace:

void ThreadPool<T>::DequeueCallAndExecute() {
for (;;) {
Call this_call;
// Wait until either the queue contains an element or this class is shutting
// down.
{
absl::MutexLock l(&lock_);
auto const has_calls_or_shutdown = [this] {
return shutdown_ || !calls_.empty();
};
lock_.Await(absl::Condition(&has_calls_or_shutdown));
if (shutdown_) {
break;
}
this_call = std::move(calls_.front());
calls_.pop_front();
}
// Execute the function without holding the |lock_| as it might take some
// time.
internal_thread_pool::ExecuteAndSetValue(this_call.function,
std::promise<T>& promise) {
promise.set_value(function());
return pile_up->DeformAndAdvanceTime(current_time_);
Status PileUp::DeformAndAdvanceTime(Instant const& t) {
absl::MutexLock l(lock_.get());
Status status;
if (psychohistory_->back().time < t) {
DeformPileUpIfNeeded(t);
void PileUp::DeformPileUpIfNeeded(Instant const& t) {
if (apparent_part_rigid_motion_.empty()) {
RigidMotion<PileUpPrincipalAxes, NonRotatingPileUp> const pile_up_motion =
euler_solver_->MotionAt(
t, {NonRotatingPileUp::origin, NonRotatingPileUp::unmoving});
for (auto& [part, actual_rigid_motion] : actual_part_rigid_motion_) {
actual_rigid_motion =
pile_up_motion * RigidMotion<RigidPart, PileUpPrincipalAxes>(
rigid_pile_up_.at(part),
PileUpPrincipalAxes::nonrotating,
PileUpPrincipalAxes::unmoving);
}
return;
}
// A consistency check that |SetPartApparentDegreesOfFreedom| was called for
// all the parts.
// TODO(egg): I'd like to log some useful information on check failure, but I
// need a clean way of getting the debug strings of all parts (rather than
// giant self-evaluating lambdas).
CHECK_EQ(parts_.size(), apparent_part_rigid_motion_.size());
for (not_null<Part*> const part : parts_) {
CHECK(Contains(apparent_part_rigid_motion_, part));
}
Instant const& t0 = psychohistory_->back().time;
Time const Δt = t - t0;
MechanicalSystem<Apparent, ApparentPileUp> apparent_system;
for (auto const& [part, apparent_part_rigid_motion] :
apparent_part_rigid_motion_) {
apparent_system.AddRigidBody(
apparent_part_rigid_motion, part->mass(), part->inertia_tensor());
}
auto const apparent_angular_momentum = apparent_system.AngularMomentum();
auto const apparent_inertia_tensor = apparent_system.InertiaTensor();
auto const apparent_inertia_eigensystem =
apparent_inertia_tensor.Diagonalize<PileUpPrincipalAxes>();
Rotation<PileUpPrincipalAxes, ApparentPileUp> const apparent_attitude =
apparent_inertia_eigensystem.rotation;
// The motion of a hypothetical rigid body with the same moment of inertia and
// angular momentum as the apparent parts.
RigidMotion<PileUpPrincipalAxes, ApparentPileUp> const
apparent_pile_up_motion(
RigidTransformation<PileUpPrincipalAxes, ApparentPileUp>(
PileUpPrincipalAxes::origin,
ApparentPileUp::origin,
apparent_attitude.Forget<OrthogonalMap>()),
apparent_angular_momentum / apparent_inertia_tensor,
ApparentPileUp::unmoving);
// In a non-rigid body, the principal axes are not stable, and cannot be used
// to determine attitude. We treat this as a flexible body, and use a part to
// propagate the attitude: its attitude at |t0| is used, together with its
// orientation with respect to the new principal axes (from apparent
// coordinates at |t|), to define the attitude at |t0| of the new principal
// axes.
// We choose the part which rotates the least with respect to the
// (instantaneous) principal axes to define the attitude.
AngularFrequency reference_part_proper_ω = Infinity<AngularFrequency>;
Part* reference_part = nullptr;
std::optional<OrthogonalMap<RigidPart, PileUpPrincipalAxes>>
reference_part_orientation_in_principal_axes;
for (auto const& [part, apparent_part_motion] : apparent_part_rigid_motion_) {
RigidMotion<RigidPart, PileUpPrincipalAxes> const
part_motion_in_principal_axes =
apparent_pile_up_motion.Inverse() *
apparent_system.LinearMotion().Inverse() * apparent_part_motion;
auto const part_proper_ω =
part_motion_in_principal_axes.angular_velocity_of<PileUpPrincipalAxes>()
.Norm();
if (part_proper_ω < reference_part_proper_ω) {
reference_part_proper_ω = part_proper_ω;
reference_part = part;
reference_part_orientation_in_principal_axes =
part_motion_in_principal_axes.orthogonal_map();
}
}
OrthogonalMap<RigidPart, NonRotatingPileUp> const
reference_part_initial_attitude =
CHECK(other != nullptr);

@pleroy
Copy link
Member

pleroy commented Oct 7, 2020

The bug appears to be in Diagonalize. (Surely the last bug in this code.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
2 participants