Researchers Launch the Phonon Olympics to Benchmark Leading Open-Source Thermal Conductivity Tools

The world of computational materials science just saw something unusual but incredibly useful: a friendly competition nicknamed the Phonon Olympics, where developers and expert users teamed up to benchmark three of the most widely used open-source software packages for thermal conductivity calculations. Although there were no medals involved, the project delivered something far more valuable—clarity on how consistently these tools perform when pushed through the exact same workflow.

This multi-year effort was coordinated by Mechanical Engineering Professor Alan McGaughey of Carnegie Mellon University, who brought together six teams to evaluate how well three major open-source packages agree with one another: ALAMODE, phono3py, and ShengBTE. Each of the three tools had two teams working on it—one composed of the software’s own developers and another made up of seasoned expert users who were not part of the development team.

The goal was simple but important: determine whether these packages provide consistent and reliable predictions when used properly. Researchers rely heavily on these tools to understand how heat moves through materials—a crucial property for designing next-generation electronics, energy devices, and thermoelectric systems. But until now, users lacked any large-scale benchmarking effort to confirm that different codes would give comparable results.

What Exactly the Teams Tested

The participants ran benchmark calculations for four different materials, chosen to cover a variety of structures, complexities, and thermal behaviors:

• Germanium (Ge)
• Rubidium bromide (RbBr)
• Monolayer molybdenum diselenide (MoSe₂)
• Aluminum nitride (AlN)

For each material, all teams used the same harmonic and cubic force constants, ensuring that differences in the results would reflect differences in the codes or workflows—not differences in the input data.

Two computational approaches were used:

• The relaxation time approximation (RTA), a simplified method for solving the phonon Boltzmann transport equation.
• The full PBTE solution, which accounts for full phonon-phonon interaction complexities.

The project also included detailed comparisons such as:

• How thermal conductivity changes with temperature.
• How much different frequency ranges contribute to total thermal conductivity.
• How modeling choices influence accuracy and computational cost.

These comparisons were especially highlighted in the case of isotopically pure 70Ge, where the teams produced both temperature-dependent results and thermal conductivity accumulation curves.

What the Benchmark Revealed

Across all teams and all tools, the calculated thermal conductivities fell within 15% of the mean value for each material. That level of agreement is encouraging—especially considering that thermal conductivity predictions are notoriously sensitive to small changes in modeling choices, supercell sizes, symmetry handling, and numerical settings.

McGaughey noted that the agreement between the tools was actually better than expected, emphasizing that well-executed workflows produce consistent predictions even when using different software.

The project didn’t stop at comparing numbers. The teams also spent time documenting best practices, covering essential aspects such as:

• How to model atomic interactions.
• How to choose appropriate displacement amplitudes.
• How to construct digital material samples.
• How to determine balance between computational cost and accuracy.

Beginners sometimes assume they can just download a code, run a script, and automatically get reliable results. This benchmarking effort reinforces that user choices matter, and careful setup is critical to accuracy.

Why This Benchmarking Effort Matters

Thousands of researchers rely on these tools to study thermal transport—a central property in many cutting-edge applications, including:

• Miniaturized electronics where heat dissipation is a bottleneck.
• Thermoelectric materials that convert heat into electricity.
• Semiconductor design.
• High-temperature insulating materials.
• 2D materials research.

Having a clear picture of how consistently different tools behave provides the community with trust. It helps new researchers avoid mistakes, gives experts a standardized reference, and sets the stage for improved workflows and better-validated research.

The project also highlighted the importance of long-term community effort. Over four years, 17 researchers contributed, remaining committed through major disruptions like the global pandemic. Their collaboration demonstrates how important reliable open-source scientific tools have become.

More About the Tools Involved

ALAMODE

This package focuses on anharmonic lattice dynamics, which is essential for modeling phonon interactions that control thermal conductivity. It supports both harmonic and cubic force constant calculations and integrates well with first-principles workflows.

phono3py

A widely used Python-based tool, phono3py calculates phonon lifetimes, three-phonon scattering processes, and thermal conductivity by solving the Boltzmann transport equation.

ShengBTE

ShengBTE specializes in solving the full phonon BTE, providing highly accurate predictions for bulk crystalline materials and becoming a preferred choice in many thermal transport studies.

A Quick Refresher: What Is Thermal Conductivity?

Thermal conductivity measures how effectively a material transfers heat. At the atomic scale, heat in most solids is carried by phonons—quantized vibrations of the crystal lattice. How phonons scatter, interact, and move determines a material’s thermal behavior.

Key factors affecting thermal conductivity include:

• Crystal structure
• Bond strength
• Defects and impurities
• Isotope composition
• Dimensionality (e.g., bulk vs. monolayer)

Open-source packages like ALAMODE, phono3py, and ShengBTE calculate thermal conductivity from first principles, starting from quantum-mechanical descriptions of atomic interactions. These methods let scientists predict thermal behavior before a material is even synthesized in a lab.

Why Benchmarking Matters for New Users

For someone new to phonon calculations, the learning curve can be steep. Numerical sensitivity means that poor settings can lead to misleading results. The Phonon Olympics helps new users in several ways:

• It shows that accurate results are achievable when workflows are followed properly.
• It provides clear best practices for setting up phonon calculations.
• It reinforces that differences in results often come not from the code itself, but from how it is used.
• It guides users in choosing between RTA and full BTE approaches based on their accuracy needs.

How This Benchmark Will Influence Future Work

This study opens doors for:

• More standardized workflows across research groups.
• Better documentation and guidelines for new researchers.
• Improved versions of existing codes based on lessons learned.
• Future benchmarks that include more complex scattering processes, such as four-phonon interactions.
• Larger datasets of reference thermal conductivities for materials discovery and machine learning.

The Phonon Olympics may not have awarded gold, silver, or bronze, but it gave the community something more practical: clear evidence that these open-source tools can be trusted when used carefully, and a roadmap for improving reliability even further.

Research Reference:
Phonon Olympics: Phonon property and lattice thermal conductivity benchmarking from open-source packages
https://doi.org/10.1063/5.0289819