In 2018, the realm of quantum computing was buzzing with excitement. Numerous advancements in quantum hardware were emerging, accompanied by various claims about their remarkable potential.

However, one significant challenge was that it was difficult for those outside the field to evaluate these claims or to compare the performance of different quantum computers. To address this issue, I developed a game called Quantum Awesomeness, designed to provide hands-on experience with quantum hardware through puzzles.

Fast forward to 2024, and the scenario from 2018 appears quite similar. New quantum devices continue to emerge, and there remains a need for the public to grasp the advancements made in the last six years. Thus, I revisited my old software for puzzle generation and applied it to the latest quantum devices.

This was a brief and initial set of evaluations. Although I identified numerous opportunities to enhance the process, I opted to use the older version without making elaborate modifications. This choice ensures a straightforward comparison of IBM Quantum devices from 2018 to 2024, specifically regarding the puzzle generation task I will describe.

To share these findings, I organized a presentation. For those unable to attend, I have transformed it into a blog format, including slides with explanations for each.

This slide serves as the introduction. I appreciate your attendance! The work showcased here originates from my time at IBM, just last month. A note on the images: All are published with permission from IBM Corporation ©.

Although I've already shared the background in this blog's introduction, this slide provides an opportunity to link to my earlier blog post about this project and the accompanying detailed paper on the puzzle creation process.

Before delving into qubits and entanglement, let's focus on the puzzles themselves. As illustrated in the examples, they resemble a grid of numbers.

This grid is familiar to quantum computing experts, but we will focus on the numbers. Each number corresponds to, or closely resembles, one of its neighbors, creating a series of pairs. The objective is to identify these pairs based on the numbers displayed.

The game features multiple rounds that progressively increase in difficulty. This heightened challenge arises from the numbers deviating from their ideal values, making the pairing process more complex. This is evident in the Round 4 example, where the numbers require careful consideration.

Next, we will examine how the game generates these puzzles, utilizing a quantum computer for number creation.

The grid represents the quantum device, with colored circles indicating its qubits. The grid connections signify which qubit pairs can undergo entangling operations. For those unfamiliar, a qubit is the quantum equivalent of a bit and serves as the fundamental building block of quantum technology. Entangling operations form the basis of quantum software, providing players valuable insights about the device.

To create a puzzle, we utilize these entangling operations. Initially, the game randomly selects a set of non-overlapping pairs, followed by random angles for each pair. Subsequently, quantum gates are applied to each pair using the corresponding angle, resulting in a specific set of entangled states.

While this process is intriguing, we need to observe the entanglement by measuring the qubits, which forces them to adopt clear values of 0 or 1. The entangled states we generate exhibit two notable characteristics during measurement:

1. The probability of receiving an output of 1 is consistent across each qubit and depends on the selected angle for the pair.
2. The outputs will always be in agreement.

Initially, we focus on the first characteristic. By repeatedly running the process, we calculate the probability of a 1 for each qubit, convert these probabilities into percentages, and utilize them as the numbers in our puzzle. For example, if two colored circles display the number 20, this indicates that an entangled pair was created with a 20% chance of both qubits outputting a 1.

In Round 2, the puzzle creation process closely resembles that of Round 1, but with additional steps that introduce more opportunities for error. These imperfections result in numbers deviating from their intended values, thus increasing the challenge.

Specifically, the setup for Round 2 involves the following steps:

1. Apply the gates that create the entangled pairs from Round 1.
2. Apply gates to undo the entangled pairs from Round 1.
3. Select new random pairs and angles for Round 2.
4. Apply the gates that create the entangled pairs for Round 2.

This logic continues for subsequent rounds, where each new round involves recreating and then undoing all prior rounds before concluding with the desired round.

In practice, the game performs a slightly different approach than described. Steps 1 and 2 are partially combined to reduce the number of entangling gates needed. By doing so, we minimize imperfections, especially since these gates were a significant source of errors in 2018. Although the current quality has improved, I chose to maintain this approach for consistency.

I developed these puzzles to provide players with insight into quantum devices, yet the allure of calculations and graphing proved irresistible.

Here's what I analyze: The pairs consist of two identical numbers, so I assess their similarity by calculating the average difference across all pairs, which I term the fuzziness of the numbers.

Each pair is also associated with a randomly selected angle that determines the displayed numbers. Conversely, we can infer the angle by analyzing the numbers mathematically. I again calculate the average differences, referring to this as the difference.

Lastly, I evaluate the puzzles' solvability. Drawing from my background in quantum error correction, I utilized an algorithm based on minimum weight perfect matching (MWPM) to gauge how many pairs it successfully identifies, which I label the matching success.

Having established the metrics for our analysis, let's proceed with the graphs. Here are simulation results where a classical computer emulates a quantum computer. The puzzles originate from a 5-qubit device, facilitating easier simulation. I also introduced imperfections, assigning each process element a probability q of error. By conducting simulations with varying q values, we can observe the impact of errors on results. The x-axis of the plots indicates the number of rounds.

The most striking observation is in the fuzziness plot. Initially low in Round 1 due to minimal errors, fuzziness begins to rise in the following rounds as errors accumulate. However, as noise increases in later rounds, each qubit's output approaches a random 50% probability, resulting in a decrease in fuzziness for unfortunate reasons.

Between the low fuzziness of Round 1 and the peak, I identify what I call the Peak of Doom, which marks a critical point where performance declines significantly.

The location of this peak varies depending on the error strength. For q < 5%, the peak is unobservable. At q = 10%, it appears at Round 4, while at q = 20%, it surfaces as early as Round 2.

Now, let's examine real device results. This data comes from a device originally named ibm_qx5, a 16-qubit machine from 2017, accessible via the cloud. This was among the first devices named after a location, a tradition now followed by IBM Quantum devices. Although these 16 qubits were situated in upstate New York, it was renamed ibm_rueschlikon after the research facility in Switzerland where I began working shortly thereafter.

Two result sets are presented, in blue and orange. The blue results reflect raw values, directly utilizing the percentages for achieving an output of 1 for each qubit. The orange results indicate an attempt to refine these values based on the expected correlations between qubit pairs, detailed in the original paper.

Both data sets convey a similar narrative. The device performs admirably but struggles beyond Round 2, as the puzzles become increasingly challenging to solve. Random guesses yield correct pairings approximately 40% of the time, mirroring our algorithm's performance from Round 3 onward, indicating that the numbers offer limited guidance regarding qubit behavior.

However, the focus of this project is on the actual puzzles rather than just graphical representations. Let's explore some examples.

The Round 2 example, with and without cleanup, illustrates the challenge. While the correct solution may seem plausible, it lacks clarity for such an early puzzle.

Moving to Round 10, the clarity remains relatively intact, though the qubits in the upper right corner exhibit signs of accumulating errors, foreshadowing future difficulties.

Now, let's observe the situation in Round 50.

In this round, the numbers are gravitating toward 50%, indicating a significant introduction of randomness due to errors. Nevertheless, some signal remains, allowing us to identify likely pairs. The circled pairs differ by no more than 1% and show notable disagreement with neighboring values, providing a strong starting point for solving.

These examples represent only a few rounds from a single game. Additional games and samples can be explored through the provided data.

In conclusion, I revisited the quantum devices I had access to back in 2018. I found that those early quantum computers struggled to handle even relatively simple tasks, primarily due to the extensive time spent applying gates that should not have impacted the initial system state.

In 2024, I repeated the process on contemporary devices and observed significantly improved results. With over 100 qubits and a depth of 100 entangling gates, we now truly experience Quantum Awesomeness!

IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of **International Business Machines Corporation*, registered in numerous jurisdictions globally. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks can be found on the Web at "IBMCopyright and trademark information" at www.ibm.com/legal/copytrade.shtml.*

JRW was partially sponsored by the Army Research Office under Grant Number W911NF-21–1-0002. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Office or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.