okay so first of all thank you very much for attending this talk and welcome to video based analysis extracting cryptographic keys from video footage for devices power LED I will start by introducing ourselves my name is Ben I'm a Black Hat board member I also do some freelancing in a postdoc at Cornell Tech I have a PhD in security and privacy and together with me is Ofec which is a master of science student at the Ben Gurion University of the negative this work relies on two papers that we have recently published which are video based analysis and optical analysis you can find them online if you just look for them and moreover no prior knowledge of cryptography is required to understand this talk we try to keep it as simple as possible so the entire audience will be able to enjoy it some of the implement some of the specifics of the implementations are not I will not discuss them in the talk you will be able to find them in the papers now with that in mind let's have some fun so here is a question for the audience what do you associate with the term cryptanalysis now some of you probably associate servers and quantum computers and data centers and in general you probably tend to associate high level of computing capabilities with the term cryptanalysis others might associate spatialized hardware for example oscilloscopes as the hardware that is needed to conduct crypt some spatial agencies might appear as well. However, I tend to believe that the vast majority of you do not associate this iPhone with the term cryptanalysis because smartphones in general have weak computing capabilities. They are very popular devices. They cannot be used to apply any complex attack and the vast majority of you do not consider yourself as an entity who owns a smartphone as somebody who is interested in recovering cryptographic keys. And here is our message for today, okay? Think again. By the end of this talk, you will understand that power LEDs pose a great risk to information confidentiality and video cameras, whether they are security video cameras or video cameras of a smartphone, provide the needed infrastructure to exploit this risk. Now, by the end of this talk, we will discuss on how to recover cryptographic keys from a device by using or by obtaining a video footage of its power LED. But first of all, I would like to thank you for joining us today. First, let's try to answer where did the idea of using a video camera to recover secret keys from a device's power LED come from? Now, this is actually takes us to 2014 where the visual microphone was published by a group of researchers from MIT. These guys were able to demonstrate a speech recovery technique using a video camera where they analyzed the movement of an object. It was demonstrated on various objects, but the most iconic demonstration was the one with the bag of chips. And they were able to analyze the movement of a bag of chips from the video footage and recover the speech played nearby the bag of chips, okay? And they demonstrated it with the use of two types of video cameras. The first one was a professional video camera, a high frequency video camera that is able to provide 20,000 frames per second. And the second one was a regular video camera. And they exploited the rolling shutter in order to do so. And we will discuss the rolling shutter later on in this talk. In 2016, I started my Ph.D. And a few years later, we published the LAMPFON ATT&CK. The LAMPFON ATT&CK is a method to recover speech by using a video camera. And this is the first example. By using a photo diode. Now, a photo diode is the sensor that you can see on the right side. Okay? It's an optical sensor which converts light into electricity. Okay? Not for pictures, but for electricity. And then it needs to be digitized with an A to D. We show that attackers can analyze the movement of a desktop light bulb such as the one that you can see in the picture by by obtaining optical measurements using the photo diode. And then to re-cover the speech played by the speakers that you can see. Uh, that are placed nearby the, uh, desktop. Okay? And we demonstrated speech recovery from very distances up to 35 meters away. You can see the experimental set-up at the bottom. The photo diode was mounted into a telescope. The telescope was directed to the light bulb that was placed on top. ten. I call this CAT, because it has a similar on top of the desktop and the speakers were used in order to play the speech. Now I want to play to you the speech recoveries in order to convince you that you are able to recover speech at very high quality. The guys from the audio, can I play it? Okay. Let's try. We will make America great again. This is the original. This is from 15 meters. 25 meters away. Now the last one was recovered 35 meters away from the light bulb. And very much like the original, this was recovered from light measurements obtained via the photodiode that was directed to the rotating or to the vibrating light bulb. Now interestingly, in one of the experiments that we conducted, we found that not only we can recover speech from by obtaining optical measurements from the light bulb, we can also recover speech from the power LED that we used, from the power LED of the speakers that we used in order to recover speech from the light bulb. We used to project the sound nearby the light bulb. Okay? And this is actually led us to write or to publish an additional research that we named the Glom attack. Now the Glom attack is the exact same technique. Okay? We use, we recover speech using optical measurements obtained by an optical sensor, the photodiode. But in this specific case, we analyzed the speaker's power LED intensity, okay, when speech is played by the speakers and recovers the speech. So, this is the Glom attack. And again, you can see the exact same flat model. The photodiode is mounted to a telescope. The telescope is directed towards the power LED. You can see it in here. It's the green LED of the speakers. And we recovered again speech at very high quality. I will play to you the speech recoveries again. This time we recovered from five meters up to 35 meters away. Okay. As you can see, the speech or the quality of the speech deteriorates with distance. But then again, it was recovered by optical measurements obtained by the photodiode and optical measurements at the... optical measures taken from the power LED of the speakers. Now, this is very surprising in a way because probably some of you wonder, why can the intensity of a speaker's power LED can even be used to recover speech? And in order to answer it, we actually conducted an experiment where we took the USB speakers and played the frequency scan from the USB speakers between zero to four kilohertz. Now, we obtained... two traces. The first trace was a power trace that we obtained via an oscilloscope that was connected to the USB connector of the speakers. And the second trace that we obtained was an optical trace that was obtained by directing the photodiode towards the LED of the USB speakers. Okay? You can see them at the bottom. Now, we made very interesting observations, but in the very but I think that the most interesting observation among them was the fact that you can see that the power consumption correlates with the intensity of the power LED of the speakers, okay? Now this is very nice and very interesting, but is this correlation between the intensity of the power LED and the power consumption can also be seen in additional speakers or is it just limited to this specific model that we've used? So then again, we conducted a few additional experiments. We took various, we actually repeated the exact same experiment, but with different models of speakers. Among them there are Sony speakers, Google's Assistant, there is also JBL and Creative. You can see that we took the best one among them, okay? You can see that the correlation appears in each and every one of them. This is, by the way, spectrograms that were obtained from the optical measurements. And as you can see, the intensity of the power LED of various speakers correlates with the power consumption in the range of zero to four kilohertz. This is what we consider the universal phenomenon. Now some of you still probably wonder why do we see this correlation? So the fact is that in various, in various electrical circuits, the integrated power LED is connected directly to the power line, okay? Moreover, dedicated means intended to decouple the correlation between the power consumption and the optical and the power correlation are either not integrated to the circuit at all, or in some cases they are integrated, but they are ineffective, okay? And as a result, the power consumption of the device, which is essentially the power supply to the integrated power LED, affects the intensity of the LED. And due to this fact, the optical measurements reflect the power consumption of the device, okay? Now, at this specific point in time, I decided to leverage these findings to conduct cryptanalysis. And the idea was to apply the previously suggested cryptanalytic attacks that first relied on power traces in order to conduct them. This time we wanted to apply them using optical measurements obtained from the power LED. And this actually led us to optical cryptanalysis. It is a paper, it's called Optical Cryptoanalysis. And this is a paper, again, it is about to be presented at CCS, but you can find it online. Optical cryptanalysis is a key recovery technique to recover the key from a device using optical measurements obtained from the power LED of the device using a photodiode, okay? As you can see, we directed the photodiode towards the LED of a Raspberry Pi. And one of the first observations that we made is that The intensity of the power LED of various devices, it's not only Raspberry Pi, there are additional devices as well. So the intensity of the power LED correlates with the power consumption of the device in much wider spectrum that we initially thought. We analyzed zero to four kilohertz, you can see that in this specific case, it reaches up to 500 kilohertz, which was the upper limit of our equipment. The potential is actually much greater than 500 kilohertz. Interestingly, we conducted an additional experiment where we took GnuPG, run it on the Raspberry Pi, and extracted and obtained optical measurements from the power LED using the photodiode. As you can see in the spectrogram that appears in the right side. We can distinguish between the decrypt operations and the slip operations of the device, which we consider as an ideal. And moreover, as you can see, we can detect the beginning and the end of the cryptographic operations performed by the CPU. This is very bad for information confidentiality. This is very bad because it now opens a door to conduct the pre-testing. The previous timing cryptanalytic attacks that were demonstrated in the past, but in this specific case, they can now be demonstrated using optical measurements obtained from the power LED of the, obtained by the photodiode that was directed to the power LED of the device. And we used this understanding in order to recover RSA, ECDSA, and PSI keys, again, you can find this information in the paper. Now this is very interesting, okay? Having the ability to recover cryptographic keys using optical measurements instead of power traces is something which is very interesting. However, the primary disadvantage of this entire method is the fact that photodiodes aren't commonly used sensors, okay? The vast majority of you do not own a photodiode, or at least the photodiode that you have. So you can't use a photodiode that can be used to recover these cryptographic keys. Moreover, in order to obtain optical measurements from a photodiode, attackers must connect them to a dedicated A2D in order to sample the electricity, the output of the autodiode, okay? And the idea we had back then was to go from a photodiode to a video camera, okay? Instead of using a photodiode, we will use a video camera to obtain the optical, to obtain the video footage from the power LED of the device. Now with that in mind, I will let Ofik to discuss the threat models and the rolling shutters. Okay. So let's discuss the threat models. Our objective is to perform cryptanalysis and recover secret keys. From a device's power LED, using video cameras instead of photodiodes. Now we have two different threat models. The first one is closed video acquisition. Closed video acquisition uses a smartphone's video camera. It targets any type of power LED and requires physical access to the device. The second threat model is over the internet video acquisition. It uses an internet-connected video camera. It targets only type 2 power LEDs, which turn on when an operation is executed and turn off when the operation is stopped being executed and are commonly used in smart code readers. And it can be applied remotely over the internet using a hijacked video camera. So far, we have used photodiodes, which are analog sensors that can be sampled at a few gigahertz. Video cameras can record 60 or 120 frames per second. A sampling rate of 60 frames per second is not sufficient to perform cryptanalysis. Or is it? We used to think that a picture is the result of one atomic snapshot taken by the video camera. However, 99% of the video cameras use a rolling shutter to record the video. A single picture is the result of one atomic snapshot taken by the video camera. Any other picture is the result of multiple snapshots taken at different times by the video camera. A video camera uses a rolling shutter to scan an object vertically or horizontally. And combines the scanned pieces into a single frame. Only few rows are captured atomically each time. And the entire picture consists of multiple snapshots of an object. Here you can see an example of a picture captured using a rolling shutter. The picture has a single frame. On the left there is a moving object and on the right there is a captured image. As you can see because each a couple of rows are captured uh in different times, distortions appears in pictures of fast moving objects. Such distortions should not appear if the picture was taken using one atomic snapshot of the object. Now let's describe an experiment we conducted to visualize the effect of the rolling shutter. First we programmed an Arduino Uno to turn its power led on and off every 250 microseconds. And here's what we can see. Now we cannot see the LED turns its power on and off at 4 kilohertz because the sampling rate is only 60 frames per second. Now let's see what happens if the LED turns its power on and off every 250 microseconds. Second, an extra lens was used to direct the camera of a Samsung Galaxy S22 Ultra to the Arduino's power LED so that the video of the LED fills the entire frame. And here's what we can see. Now because the LED turns its power on and off few times in each frame there are red rows that were captured when the LED was turned on and black rows that were captured when the LED was turned off. And this is why we can see those black and red stripes across each frame. So by filling the frame with the video of the LED we exploit the video camera's rolling shutter to increase the number of measurements obtained at different times. This allows us to detect the 4 kilohertz flicker. Theoretically, by exploiting the video camera's rolling shutter we can increase the sampling rate by 3 orders of magnitude. From the FPS rate, 60 measurements per second, to the rolling shutter rate, 60K. In reality, there is a delay, a transition time, between consecutive frames. The transition time between frames is a period of time during which no object is captured by a video camera. You can see it here in the figure but that between two consecutive frames there is a transition time. This should be taken into account when analyzing the video footage of the power LED. Now, I will let Ben to discuss the ACDSA key recovery. Now, I will let Ben to discuss the ACDSA key recovery. Okay, so before we will discuss the ACDSA key recovery or in order to discuss the ACDSA key recovery, we need to briefly discuss the Minerva attack. The Minerva attack introduced or published three years ago and one of the most interesting revelations that researchers did was the fact that they found that various cryptographic libraries apply some runtime optimizations in the code. Okay, and I refer to the code of the elliptical of 256 R1 implementation of the ACDSA. And they found the number of iterations of the main loop is actually determined by the number of leaving zeros in the nonce. Okay, and as a result it created the time dependency between the number of leading zeros in the nonce and the execution time of the main loop and the the EC-DSA signing time. And the Minerva attack introduced a technique to recover the EC-DSA key by analyzing the EC-DSA signing measurements of a set of EC-DSA signatures. You need to have about 4,000 signatures in order to recover and the associated EC-DSA signing time in order to recover the complete key. Now the original Minerva attack required the attackers to obtain measurements from the CPU. The timing measurements obtained by querying the CPU. And essentially it required the attackers to compromise the host machine. The authors were unable to demonstrate the attack remotely over the internet due to the noise. The Minerva attack is very sensitive to errors in the timing measurements. You are about to see it now demonstrated over the internet. Without getting into too much specifics and details regarding the Minerva attack, you can think about it as a black box. The black box receives a set of signatures and their associated signing time and returns the EC-DSA key out of them. So essentially what we need to answer is how can the EC-DSA signing time of a signature be estimated from video footage. If we will be able to answer it, we will be able to recover the complete EC-DSA key using video footage. Now this was the experimental setup in our experiment. We used it, we placed an internet connected video camera and directed it to the power led of the smart card reader. Inside the smart card reader there was a smart card with an EC-DSA key. And we wanted to recover the EC-DSA key from the smart card using video footage of the power LED of the smart card reader by using this internet connected video camera that was placed 16 meters away from the camera. From this desktop. Okay, and this is the experimental setup. I want to show to you the video. You can see on the left side this was the smart card reader that we use. We actually focus on the right power LED of it. And let me show to you. Now you can see that we zoom into the power LED of the smart card reader. It is located on the right side. card. And interestingly, the smart card actually provides you with indication regarding whether the smart card reader actually provides you with indication regarding the smart card whether an operation actually taking place or not. And this is done by changing the color of the LED between blue or basically turn it off. Okay. Any time it's blue, the smart card is in idle. Any time it's off, the smart card is being used to sign. So as you can see from the video on the right side, there are differences between black and blue colors, okay, which indicate whether the smart card is currently being used to sign on stuff or not. So as you can see, this actually the picture on the left side is actually the result of averaging the each row in a frame to a single volume and arrange them in a time series, okay? And we did it for the red, green, and blue channels. You can see that we can distinguish between the sign operations where the power LED is off and the idle operations where the color of the power LED is blue based on the blue channel. Now we need to answer how can we calculate the signing time out of it? Now in order to calculate the time out of the power LED, we need to calculate the EC-DSA time of the signature, we extract the series frames associated with each signature from the video, okay? You can see examples of them on the bottom of the slide. Now we kept series that contained the indication of the switch between the beginning and the end of the signing. Meaning that we kept series that the beginning and the end of the signing were kept in the beginning of the signature. Now we can see in the first series, where you can see in this specific case, this is the series that you can see on top, okay? Each series that missed or lacks one of these indications regarding the switch between the beginning of the signature and the end of the signature, we filtered it off, okay? You can see that how the in the frames that appear in the first series, the color is changed inside the frame between blue and black and vice versa. Okay? This is what we considered as the indication that the operation started and ended. Okay? And these are only the type of series that we kept. Now, first of all, we calculated T1, which T1 in our case is the execution time of all of the full black frames that you can see in here. This is actually done just by counting the number of full black frames and multiplying them by this constant. This gives you T1. T2 is the execution time of the first frame, okay, in which the color changed between, as you can see in here, between blue to black. So we calculated this as follows. We counted the number of rows that are associated with the sign, which you can see in here. They are in black. And divided them by the total number of rows in the frame. So this is the time. Okay? This actually gives us the relative time in the frame. We multiply it by the scanning time and added the transition time. Now, so far, Ofik discussed about the scanning time and the transition time, but he did not mention how you can determine the scanning time and the transition time. So let's discuss how we can find S, which stands for the scanning time, and T, which stands for the transition time. So I remind you the experiment that we did at the beginning. With the flickering LED of the Arduino. You can just take one of the you can repeat the exact same experiment with the flicker. Take only one of the frames from the video. Count the number of changes between black and red that you have in the frame and multiply it by the time that the flicker was on and off. In this specific case, the on and off periods were equal. Okay? And by doing so, this will gives you the scanning time of the video camera. And the beauty of it is then when you get the scanning time out of this experiment, you get the transition time for free by deducting the scanning time out of this constant. Now, with that in mind, returning to T2, you now have S and T and everything needed to calculate T2. Now, T3 is actually calculated at the exact exact same way, okay? But in this specific case, we do not add the transition time because the ECDSA sign stopped or ended in the middle of the frame, so the transition time happened after the frame, so there is no need to add it to this specific, to T3. And the sign time is actually the sum of T1, T2, and T3. Now, the beauty of it is that if you will do it to enough signatures, for about 4,000 signatures, you can see that this is the result that we got. We were able to recover the ECDSA key out of it using the Minerva script that the orders were published, again, using an internet-connected video camera that was directed to a power LED of a smart card reader. Thank you. Okay. Now, moreover, there are many additional smart card readers available to purchase on Amazon that are vulnerable to this attack, and the distance between them and the video camera might vary based on the intensity of their own power LED, but we were able to recover the same ECDSA key from the smart card, inserting it to five additional smart card readers. Again, all of them are available online. Okay. Now, let's discuss recovering a side key from a device. And again, recovering a side key from a device, so in order to understand how to recover a side key from a device, we need to briefly discuss the Herzblitz attack. The Herzblitz attack introduced a year ago. They also presented their talk at Black Hat, and they were able to... Suggest or introduce a new timing attack to recover the complete side key from a server. The Herzblitz attack, unlike the Minerva attack that I showed you before, is not the result of the implementation of the code. Okay? It's the result of the execution of the device. The code should be resilient to timing attack, but the execution actually reveals something in terms of the time that it was executed regarding the data being executed by the device. And this is happening due to the dynamic voltage frequency stabilizers. Maybe you are more familiar with a DVFS, okay, which yields different execution times based on the data that was recovered, that is being processed by the device. And the researchers were able to show that for each index of the bit, they can craft a dedicated cryptogram that relies on the indexes that they were already produced. recovered. Okay, it's an adaptive attack for those of you who are familiar with cryptography. And determine whether the value of the bit under attack is similar to the value that was recovered in the previous index or changed. Okay, and this is all done based on a timing threshold. Now, the original attack was implemented using timing measurements obtained by querying the API of a server over the Internet. They deducted the request time from the response time. And, again, due to the noise that was added by the network latency, it took a few days to recover the keys . Now, this was our attack, okay? And this is the experimental setup of our attack. We tried to recover the secret key, the side key, from the Samsung Galaxy S8. The Samsung Galaxy S8 started 8 years ago. It was theरÁ dares to beat my souha? Galaxy S8 is the smartphone that you can see on the right side. It's not the smartphone that you can see on top. The Samsung Galaxy S8 was the device that holded the side key. Okay? Now, interestingly, in this specific case, we tried to recover the secret key from a device by obtaining video footage not from the Samsung Galaxy but from the speakers, okay, the power led of the speakers that were connected to the same USB hub that was used to charge the Samsung Galaxy. And in order to make it completely outrageous, we did it with an iPhone, tried to recover the key from the Samsung Galaxy S8. On the right side, you can see the video footage of the power led of the speakers. Okay, you can see that it is green. This is actually being taken from the speakers as you can see on the left side. Between the speakers and the smartphone, you can see a lens which was used to fill the entire frame with the view of the LED. I want to show to you the video. You cannot see anything, right? It seems like completely innocent video with green color. Moreover, I will argue that you cannot see even the difference between the IDLE and the PSYC operations being performed by the Samsung Galaxy S8. Okay? They cannot be detected by the naked eye. But here is something very interesting. Okay? We executed eight consecutive iterations of PSYC operations where each iteration consisted of 1100 of PSYC operations on the Samsung Galaxy S8. say while obtaining video footage from the power LED of the connected speakers. You can see that by averaging each frame into a single value and arranging them in a time series and then again zooming into the green channel, you can see that we can detect the beginning and the end of the eight iterations by analyzing the green channel and again using this by arranging the values in a time series. Now this actually gives you the ability to calculate the iteration time based on the number of values between each two peaks that you can see in here, okay? But then again, another question that we need to answer is how accurate is this calculation? So, on the left side, you can see the distribution of the execution time based only on the first iteration, okay? We apply the attack by using only the first iteration and as you can see, it's very noisy. The two possibilities or the two cases that whether the bit have a similar value or was it changed in comparison to the previous index? You cannot set a threshold that will give you an accurate, you will be able to use it in order to accurately distinguish between the two specific cases. However, when you use instead the distribution of the execution time based on the minimum execution time among these eight iterations, what you're seeing in the middle, you can see that you can actually set a threshold. threshold in the middle that will allow you to distinguish between the two possible bits of the key index with very high accuracy, okay, 99% accuracy. Now 99% accuracy is good. It's not excellent, okay? It's not error-prone. And as a result, we will have to use an error detection and correction mechanism throughout the key recovery process in order to recover the complete key. And as you can see, this is actually what we did. We recovered the complete key, the complete side key from the device using a video footage obtained from the connected speakers. Then again, we also integrated an error detecting and correcting mechanism that was suggested by the guys that presented the Herzbleed attack. Okay. So let's briefly discuss the limitations of video-based script analysis. So first of all, the first limitation is the fact that the attack mostly targets weak IoT devices. I would say up to a level of a smartphone. I would be very surprised if somebody will take the exact same video cameras that we used and will show me that he was able to recover secret keys from a power lead of servers or power lead of laptops. Now, moreover... We discussed about it earlier. Due to the fact that there is a transition time where between each two frames where the object or the power lead isn't being captured in any frame, the sampling, the distribution of the sampling is semi-uniform. It's not exactly uniform, which is another issue that we need to resolve when recovering the secret keys. Now, let's discuss the takeaways. I think that now you are convinced that power LEDs are much more informative than you initially imagined. In some cases, attackers can use the information modulated over the power LED to recover speech. In other cases, it can be used to recover cryptographic keys, as we showed in this talk. Now, moreover, the potential of the attack is actually much greater than we showed you in this work. We focused on what we consider as ubiquitous cameras, not professional cameras. We use security cameras and smartphones, video cameras, in order to recover the secret keys. Attackers in reality can use professional video cameras that will provide a higher rolling shutter rate, a higher bit depth, and enhance zoom capabilities, and by doing so, they probably will be able to recover... Secret keys from a wider range of devices, from probably even a greater distance. And this is, I think, the most interesting insight or takeaway from this talk. We expect that more and more devices will be exposed to video-based cryptanalysis every year. Okay? This is actually the result of two interesting facts. First of all, video cameras' specifications... Are continuously improving each and every year, following Moore's law. You can see how the sensitivity of this picture has improved significantly throughout the years. Moreover, the number of functional IoT devices with weak CPU power, and I'm referring to sensors and smart cards and robotic vacuum cleaners and smartphones and even video streamers, the number of such devices... Are continuously increased each and every day. Their deployment is expected to grow throughout the next year as well. And as a result, we will have much improved video cameras throughout the years that co-located with a greater number of weak IoT devices. Okay? And due to this fact, we believe that more and more devices will be exposed to video-based cryptanalysis each and every year. Okay? This is the final takeaway. It's mostly meant for perspective. It might be easier for this audience to compromise the target device with the malware and exfiltrate the key over the Internet. Okay? Than applying the attack that I've just... The attacks that I've just mentioned. However, bear in mind that video-based cryptanalysis is intended to extract keys from non-compromised devices. Okay? And the beauty of it is that it uses popular equipment. Okay? And the beauty of it is that it uses popular equipment. Not something that we consider as specialized equipment as the equipment that I've discussed at the beginning. Now, a few other things I would like to mention. This research was recently awarded with a Pony Award for the best cryptographic attack of 2023. You can find the additional details if you will look for video-based cryptanalysis online. Okay? And with that in mind, thank you very much for attending this talk. I will be happy to take any questions from the audience. Thank you. .