Interpreting Stack Traces

POLARIS uses the cpptrace library to generate a stack trace when there are unexpected exceptions. This is a system-generated stack of functions called leading to the unexpected crash or exit:

2025-07-14 13:51:06,572 Thread: 123188177479744 | [ ERROR] Stack trace (most recent call first):
#2  0x000000000004251f at libc.so.6
#3  0x0000000000091117 at libc.so.6
#4  0x0000000000093a40 in pthread_cond_wait
                       at libc.so.6
#5  (inlined)          in void std::condition_variable::wait<polaris::World::ThreadGate::wait_at_gate() const::{lambda()#1}>(std::unique_lock<std::mutex>&, polaris::World::ThreadGate::wait_at_gate() const::{lambda()#1})
                       at condition_variable:111:8
#6  (inlined)          in polaris::World::ThreadGate::wait_at_gate() const
                       at World.h:98:24
#7  0x0000000000c6b99a in polaris::World::Start_Turning(std::filesystem::__cxx11::path)
                       at World.cpp:266:35
#8  0x00000000005a8dad in operator()
                       at Integrated_Model.cpp:444:44
#9  0x000000000057aa66 in main
                       at Integrated_Model.cpp:519:33
#10 0x0000000000029d8f at libc.so.6
#11 0x0000000000029e3f in __libc_start_main
                       at libc.so.6
#12 0x000000000057f724 in _start
                       at Integrated_Model

This stack trace gives line info (assuming we compiled with some debug symbols - a default ‘Release’ build will give us very little info from the trace), and makes it much easier for POLARIS developers and researchers with access to the source code to diagnose unexpected errors.

When that isn’t enough

If the above information isn’t enough to track down the root cause - the next step would be to run the offending model through a debugger.