In the blink of an eye, the unruly, superheated plasma that drives a fusion reaction can lose its stability and escape the strong magnetic fields confining it within the donut-shaped fusion reactor. These getaways frequently spell the end of the reaction, posing a core challenge to developing fusion as a non-polluting, virtually limitless energy source.

But a Princeton-led team composed of engineers, physicists, and data scientists from the University and the U.S. Department of Energy’s Princeton Plasma Physics Laboratory (PPPL) have harnessed the power of artificial intelligence to predict — then avoid — the formation of a specific plasma problem in real time.

In experiments at the DIII-D National Fusion Facility in San Diego, the researchers demonstrated their model, trained only on past experimental data, could forecast potential plasma instabilities known as tearing mode instabilities up to 300 milliseconds in advance. While that leaves no more than enough time for a slow blink in humans, it was plenty of time for the AI controller to change certain operating parameters to avoid what would have developed into a tear within the plasma’s magnetic field lines, upsetting its equilibrium and opening the door for a reaction-ending escape.

“By learning from past experiments, rather than incorporating information from physics-based models, the AI could develop a final control policy that supported a stable, high-powered plasma regime in real time, at a real reactor,” said research leader Egemen Kolemen, associate professor of mechanical and aerospace engineering and the Andlinger Center for Energy and the Environment, as well as staff research physicist at PPPL.

The research opens the door for more dynamic control of a fusion reaction than current approaches, and it provides a foundation for using artificial intelligence to solve a broad range of plasma instabilities, which have long been obstacles to achieving a sustained fusion reaction. The team published their findings in Nature on February 21.

“Previous studies have generally focused on either suppressing or mitigating the effects of these tearing instabilities after they occur in the plasma,” said first author Jaemin Seo, an assistant professor of physics at Chung-Ang University in South Korea who performed much of the work while a postdoctoral researcher in Kolemen’s group. “But our approach allows us to predict and avoid those instabilities before they ever appear.”

Superheated plasma swirling in a donut-shaped device

Fusion takes place when two atoms — usually light atoms like hydrogen — come together to form one heavier atom, releasing a large amount of energy in the process. The process powers the Sun, and, by extension, makes life on Earth possible.

However, getting the two atoms to fuse is tricky, as it takes massive amounts of pressure and energy for the two atoms to overcome their mutual repulsion.

Fortunately for the Sun, its massive gravitational pull and extremely high pressures at its core allow fusion reactions to proceed. To replicate a similar process on the Earth, scientists instead use extremely hot plasma and extremely strong magnets.

In donut-shaped devices known as tokamaks — sometimes referred to as “stars in jars” — magnetic fields struggle to contain plasmas that reach above 100 million degrees Celsius, hotter than the center of the Sun.

While there are many types of plasma instabilities that can terminate the reaction, the Princeton team concentrated on solving tearing mode instabilities, a disturbance in which the magnetic field lines within a plasma actually break and create an opportunity for the plasma’s subsequent escape.

“Tearing mode instabilities are one of the major causes of plasma disruption, and they will become even more prominent as we try to run fusion reactions at the high powers required to produce enough energy,” said Seo. “They are an important challenge for us to solve.”

Fusing artificial intelligence and plasma physics

Since tearing mode instabilities can form and derail a fusion reaction in milliseconds, the researchers turned to artificial intelligence for its ability to quickly process and act in response to new data.

But the process to develop an effective AI controller was not as simple as trying out a few things on a tokamak, where time is limited, and the stakes are high.

Co-author Azarakhsh Jalalvand, a research scholar in Kolemen’s group, compared teaching an algorithm to run a fusion reaction in a tokamak to teaching someone how to fly a plane.

“You wouldn’t teach someone by handing them a set of keys and telling them to try their best,” Jalalvand said. “Instead, you’d have them practice on a very intricate flight simulator until they’ve learned enough to try out the real thing.”

Like developing a flight simulator, the Princeton team used data from past experiments at the DIII-D tokamak to construct a deep neural network capable of predicting the likelihood of a future tearing instability based on real-time plasma characteristics.

They used that neural network to train a reinforcement learning algorithm. Like a pilot trainee, the reinforcement learning algorithm could try out different strategies for controlling plasma, learning through trial and error which strategies worked and which did not within the safety of a simulated environment.

“We don’t teach the reinforcement learning model all of the complex physics of a fusion reaction,” Jalalvand said. “We tell it what the goal is — to maintain a high-powered reaction — what to avoid — a tearing mode instability — and the knobs it can turn to achieve those outcomes. Over time, it learns the optimal pathway for achieving the goal of high power while avoiding the punishment of an instability.”

While the model went through countless simulated fusion experiments, trying to find ways to maintain high power levels while avoiding instabilities, co-author SangKyeun Kim could observe and refine its actions.

“In the background, we can see the intentions of the model,” said Kim, a staff research scientist at PPPL and former postdoctoral researcher in Kolemen’s group. “Some of the changes that the model wants are too rapid, so we work to smooth and calm the model. As humans, we arbitrate between what the AI wants to do and what the tokamak can accommodate.”

Once they were confident in the AI controller’s abilities, they tested it during an actual fusion experiment at the D-III D tokamak, observing as the controller made real-time changes to certain tokamak parameters to avoid the onset of an instability. These parameters included changing the shape of the plasma and the strength of the beams inputting power into the reaction.

“Being able to predict instabilities ahead of time can make it easier to run these reactions than current approaches, which are more passive,” said Kim. “We no longer have to wait for the instabilities to occur and then take quick corrective action before the plasma becomes disrupted.”

Powering into the future

While the researchers said the work is a promising proof-of-concept demonstrating how artificial intelligence can effectively control fusion reactions, it is only one of many next steps already ongoing in Kolemen’s group to advance the field of fusion research.

The first step is to get more evidence of the AI controller in action at the DIII-D tokamak, and then expand the controller to function at other tokamaks.

“We have strong evidence that the controller works quite well at DIII-D, but we need more data to show that it can work in a number of different situations,” said first author Seo. “We want to work toward something more universal.”

A second line of research involves expanding the algorithm to handle many different control problems at the same time. While the current model uses a limited number of diagnostics to avoid one specific type of instability, the researchers could provide data on other types of instabilities and give access to more knobs for the AI controller to tune.

“You could imagine one large reward function that turns many different knobs to simultaneously control for several types of instabilities,” said co-author Ricardo Shousha, a postdoc at PPPL and former graduate student in Kolemen’s group who provided support for the experiments at DIII-D.

And on the route to developing better AI controllers for fusion reactions, researchers might also gain more understanding of the underlying physics. By studying the AI controller’s decisions as it attempts to contain the plasma, which can be radically different than what traditional approaches might prescribe, artificial intelligence may be not only a tool to control fusion reactions but also a teaching resource.

“Eventually, it may be more than just a one-way interaction of scientists developing and deploying these AI models,” said Kolemen. “By studying them in more detail, they may have certain things that they can teach us too.”

The paper, “Avoiding fusion plasma tearing instability with deep reinforcement learning,” was published February 21 in Nature. In addition to Kolemen, Seo, Jalalvand, Kim, and Shousha, co-authors include Rory Conlin, Andrew Rothstein, Joseph Abbate and Josiah Wai of Princeton University, as well as Keith Erickson of PPPL.

The work was supported by the U.S. Department of Energy’s Office of Fusion Energy Sciences, as well as the National Research Foundation of Korea (NRF). The authors also acknowledge the use of the DIII-D National Fusion Facility, a Department of Energy Office of Science user facility.