start	end	text
1040	6120	Hey everyone! Welcome to my talk, Car2Cloud Talk Using MQTT for Car Hacking.
6120	10540	My name is Jamie Lightfoot. I used to work as an electrical engineer, and now I work
10540	12260	at Grimm doing car hacking.
13900	19180	Hey everyone! Welcome to my talk, Car2Cloud Talk Using MQTT for Car Hacking.
19180	23400	My name is Jamie Lightfoot. I used to work as an electrical engineer, and now I work
23400	24920	at Grimm doing car hacking.
25100	29860	This is a Car Hacking Village 101 talk, which is meant to introduce people to foundational
42800	47860	So, let's talk about how MQTT works and how you can use it.
47860	51180	Let's talk about why we're talking about it in the first place.
52900	58140	Automotive companies won't directly come out and say it, but if you've got a newer car or truck,
58140	59840	then they're probably collecting a lot of data.
59840	63740	They have information on you and your driving habits all the time.
64000	69140	Some of this is still the safety features like OnStar or other ways of remote assistance
69800	72900	where they need to know the location of your car.
73100	78960	Some of this is still this convenience. For example, you've got remote unlock or lock functionality,
78960	82640	remote start, remote alarm through like a phone app.
82740	86940	There's also over there updates so you don't have to go into a dealership.
86940	92440	Some of this connectivity is much more for the engineering company's benefit than for the consumer.
92500	97080	In a past job, I worked on a project where we aggregated truck engine and transmission data
97080	101900	and we would use that to predict future maintenance needs and ship the repair parts
101900	106120	to where we knew the trucks were going to be weeks in advance.
106320	109960	We would save the company tons of money on downtime and we'd also get them insight
110500	114400	into what kind of warranty claims were going to be coming up.
116800	122080	In all of these cases, the data somehow has to get to or from the car to somewhere else useful
122080	128540	like your OEM branded mobile app, some kind of online engineering portal, a dealership tool, or whatever.
128560	131120	And that's where MQTT comes in.
131700	138240	Most of this talk is going to be about the protocol itself, but to give you an idea of how this ties into an actual car,
138240	147640	For newer cars and trucks, we have a Telematics Control Unit, or TCU, which is the PCB in this diagram.
148840	154980	The TCU is going to be likely on the CAN bus and accept other inputs from the car.
154980	160740	For example, if there's a GPS antenna, so it has the ability to interface with the car
160740	165300	and it also has an understanding of what is currently happening in the car.
165300	172300	It'll be able to use its cellular connection to the cloud to send information in the form of MQTT messages.
172440	178120	It can also do the same thing in reverse, where it is accepting MQTT messages from the cloud
178120	184880	and then taking some kind of action within the car, for example, sending information out over the CAN bus.
186700	192060	The TCU device has to have, at a minimum, the ability to connect to a cellular network
192060	197660	or more likely networks, plural, as coverage is going to vary throughout a given area
198340	200940	and you want to be able to switch between them.
200940	208060	And then CAN transceivers are another way of interfacing with the other parts of what is happening in the car.
208260	213140	But if you have a cellular connection from the car to the cloud,
213140	217260	why would you choose MQTT over some other method of communication?
217260	225980	Like maybe you're able to send a text to the TCU device and wake it up and then have it establish an HTTP connection.
226040	233020	The reason is because cars are a bit of a difficult use case from a network and architecture standpoint.
233020	238420	You've got unreliable connectivity or maybe varying connectivity quality because you're driving around
238420	241400	and you might go through areas with poor signal.
241420	246080	So you need to have some kind of protocol that has minimal overhead and size.
246080	250860	You might want the ability to buffer messages if you're offline for a minute.
251600	257580	And you also might want some kind of method to know if your message was received or not.
258040	264560	You're probably dealing with high network latency, which means that you want to reduce overhead as much as possible.
264980	271100	MQTT clients will remain connected and they don't need to reinitiate a connection with each message.
271100	280000	Whatever architecture you use also needs to be able to scale up to millions of vehicles and handle millions and millions of messages per day.
280000	285180	Hopefully you're also considering the security of the system, but we'll talk about that more later.
286100	292600	MQTT stands for MQ Telemetry Transport or Message Queuing Telemetry Transport.
292600	300280	It came from Andy Stanford Clark and Arla Nipper who are at IBM and Eurotech respectively.
300280	306000	IBM had a MQ or Message Queuing product at the time.
306000	316260	The OASIS standard for MQTT does not use queues to describe the message communication, but that's where the name came from.
316260	322100	It was originally made to monitor oil pipelines over a satellite link.
322100	327660	So that's where the emphasis on low bandwidth, very lightweight came from.
327660	340940	MQTT.org describes it as a publish, subscribe, extremely simple and lightweight messaging protocol designed for constrained devices and low bandwidth, high latency or unreliable networks.
340960	351720	The design principles are to minimize network bandwidth and device resource requirements while also attempting to ensure reliability and some degree of assurance of delivery.
351940	357040	Given the network challenges that we talked about earlier, it seems like MQTT is a good fit.
357660	360160	So let's talk about how it works.
361020	365180	In MQTT systems, there are two types of network entities.
365180	371220	There's an MQTT broker, now called a server, but I call it a broker out of force of habit.
371220	374600	You also have one or more MQTT clients.
375900	385420	An MQTT broker is a program that receives all the messages from the clients and then routes the messages to the appropriate destination client.
385420	388000	It's pretty much like a post office.
388900	393740	All messages have to travel through the broker in order to get to the other clients.
393740	397880	A client can't send anything directly to another client.
398440	412600	A client then is a device that is either interested in receiving certain information from the broker via other clients or publishing information to the broker, which will then be sent on to the interested clients or both.
412600	419940	In an automotive environment, a client is going to be the car or specifically the TCU unit in the car.
420680	425400	And the broker is going to be some kind of cloud service that the car company has set up.
425400	433080	A mobile app might also be a client, a dashboard might be a client, but there's usually some more architecture going on.
433080	437300	So they aren't all talking unfiltered to the same broker.
438040	443320	Here are a few examples of cloud broker services that I have seen or worked with.
443320	446400	There are others. These are just a couple options.
446800	452700	To visualize brokers and clients, I made this diagram where we have a broker in the middle and a number of clients.
453020	458480	Each client can either subscribe, publish, or both.
458480	463140	What kind of behavior they do is contingent on how they try and connect to the broker.
463140	464980	We'll see more about that in a minute.
464980	471760	Earlier I said that it is a publish-subscribe method, or PubSub for short.
471760	477300	Each client can designate what information it wants to publish to the broker.
477300	482260	And each client can also specify what information it wants to receive back.
482260	486460	And then the broker just doles out the messages as intended.
486540	493100	Additionally, clients can provide forms of authentication to the broker, but they aren't directly addressable.
493100	497180	So if you have a whole fleet of trucks or cars sending out messages,
497180	504140	you probably want to be able to keep the messages separate or organized so you know which messages apply to which vehicle.
504200	506440	And that's where topics come in.
506860	511820	You can designate topics however you like. However, there are some patterns to follow.
511920	519860	Here I've got some example topic names where maybe we have a DEFCON-wide broker
519860	531620	and there's a topic called CarHackingVillage where if you subscribe to that, you get all the information being sent out related to CarHackingVillage.
532320	538520	And then if you publish to it, then everyone who is subscribed will receive that information.
539360	545820	But, because CarHackingVillage has a lot of different things going on, let's break it down into subtopics
545820	552220	where you have CarHackingVillage slash talks or CarHackingVillage CTF,
552220	561580	where if you're only interested in the talks or the CTF, you can subscribe just to that and narrow down the information that you receive.
561680	569320	Likewise, if you're publishing, you are publishing to a more narrowed down audience.
570460	574240	And then you can just keep nesting as far as you want.
574240	584440	For example, maybe in the CTF we have different team IDs and you can subscribe to your team ID and see what score you have.
585480	590920	So, if you've got people publishing to each of these individualized nested topics,
590920	595480	if you still want to receive all of the information about CarHackingVillage,
595480	604540	then you will want to subscribe to CarHackingVillage slash either pound sign or hashtag, depending on how old you are,
604540	608200	and you'll receive all of the CarHackingVillage topics.
608880	618180	If you were to subscribe to the pound sign by itself, that would function as a wildcard and give you all topics available on that broker.
618180	624460	So, for our DEF CON example, it would give you information on all the villages, all the CTFs, all the things happening.
626420	632180	But if you are a client, this is probably more information than you want or that you can handle.
632180	640220	If you're a top-level component in the architecture, you're probably interfacing with the broker in a different way to get all the information out.
641020	647640	A client can only publish to one topic at a time. You can't publish to a group of topics or a wildcard.
647640	654320	For the publishing volume, it's up to the client, but think on the order of maybe once per second.
654320	666760	So, in automotive, the topics might look like car data, and then the VIN number, and lock status or location, latitude, location, longitude, information about the engine, and so on.
667060	676680	Where you would receive back, probably JSON. All of that is defined by your system. It's kind of however you want to set that up.
676680	682300	You can also use the pound sign and plus sign to selectively handle levels.
682300	696680	So, you could do car data and then a plus sign, location, latitude, and that would get you all of the latitudes for all vehicles if that level that you're using the plus sign for typically handles VIN data.
696680	713820	You could also do car data, pound sign or hashtag to get all information. So, the lock information, location, engine, and so on for all vehicles on that broker.
714380	722620	If you are securing a MQTT broker, you want to make sure that your clients don't have wildcard privileges.
723360	736360	There's also a set of topics that all start with dollar sign SYS. These are all administrative type topics that give information on the clients, messages being sent, and so on.
736360	742860	If you are securing a broker, you want to make sure that these are turned off for all non-admin users as well.
742860	756100	Revisiting this diagram, receiving information isn't like the request response workflow that you're probably used to. You are not requesting data and then getting a response back containing that data.
756100	766820	Instead, you are subscribing to topics you want data from, and then when data is published to those topics, you receive that information.
769190	781750	After you're subscribed, you'll continue receiving information on those topics from other clients as it gets published until you either disconnect from the broker entirely or you unsubscribe from that topic.
781870	788530	There's also an optional small set of delivery assurance messages. We'll talk about that a little bit later.
788530	803250	If we revisit our earlier diagram of topics, if the first client publishes something to Topic 1, the broker is going to forward that on to Client B because Client B is subscribed to Topic 1.
803250	812510	It will also send it to Client C because Client C is subscribed to the wildcard and wants information on all topics.
813850	830670	If we talk about the MQTT packet structure briefly, all packets are going to have a fixed header that has information about the type of packet and then some flags relating to the packet type.
832150	851830	There are a number of packet types as listed here. They come in pairs. For example, you have Connect, and then Connect Acknowledgement, Publish, and then different types of Publish Acknowledgement, Subscribed, Subac, Unsubscribed, Unsubac, and then you have Ping, Ping Response, and then Disconnect.
852550	865310	The next part of the fixed header is for flags, but that pretty much only applies to the Publish message type because there are Quality of Service and other configurations that need to be set.
865310	876190	Connect, Subscribe, and Publish messages provide more information than other types of messages and need to use a variable header in addition to the fixed header to include that info.
876190	883110	For example, a Connect message is going to provide version information, keep-alive information, and so on.
883110	891310	And then after the headers, you have any kind of payload information. Typically, you're going to see JSON, but you can do whatever you want, really.
892290	899530	So, here are two examples of Wireshark captures of packets.
899530	905730	The first one is a minimal packet where it's two bytes and it is a Disconnect message.
905730	914810	And then we have a longer packet on the bottom where we are publishing hi to the CarHackingVillage topic.
917860	926400	If we look at a Subscribe message connection in Wireshark, the client is sending a Connect command.
926400	929960	The broker is going to respond with a Connect acknowledgment.
929960	939500	Then the client sends a Subscribe request, and then if it's accepted, the broker will respond with a Subscribe acknowledgment.
939500	946060	If you're doing security testing and you can't get a connection to work, I suggest two different things.
946060	959080	One is to use a dash d in your subscription command, and then the other one is to set up a test server and get an idea of what a known good connection looks like.
959240	970900	If a client has been able to successfully subscribe, after that you'll see a ping and ping response periodically between the client and the server as a keep alive signal.
970900	977040	For a Publish command, we have a Connect, a Connect, a Publish, and then a Disconnect message.
977200	982880	If we had a higher quality of service setting, then you might get an ACK or other message back.
983120	988660	And then the final publish line is from the broker to the other connected client.
989620	994680	So once again, you can use the dash d to get some more debug information.
994940	998440	There are three quality of service settings that you can choose.
998440	1008380	0 is the default, 0 is at least once, it's going to be fire and forget, there's no kind of checking or acknowledgement afterwards.
1008380	1016460	Level 1 is at least once, it's going to keep sending that message until it gets acknowledgement back.
1016620	1026320	Level 2, there's a little bit more going on, and as a result, you can ensure that only one copy is received.
1027200	1037920	At this point, I think the best thing to do is to try out a MQTT broker on your own computer to see how it works.
1037920	1044620	And to do that, I recommend that you download Mosquito, because you can do all of this on the command line.
1044620	1049540	There's very minimal setup, and I think it helps cement the ideas.
1050680	1056160	First of all, you need to go to their download page and get it set up on your computer.
1056160	1058920	If you're on Mac, you can do brew install mosquito.
1059460	1065800	Then you will need to start the service. I'm on a Mac, so I did the line shown here.
1066680	1069580	And then you want to open up two more terminals.
1069720	1072660	In the first one, you want to subscribe to a topic.
1073520	1076400	For example, I had CarHackingVillage.
1076600	1081100	And then the "-i client1 is my client ID".
1081100	1087660	You don't have to provide this. It will make one for you if you don't, but just for completeness.
1088240	1097220	And then in the other terminal window, you do a publish message to CarHackingVillage topic with whatever you want your message to be.
1097220	1104060	And then in the first terminal window where you subscribe, you should see that data come through.
1106620	1113560	Visually, that looks like starting the service. It will listen on port 1883 by default.
1113560	1120520	You subscribe, you publish, and then back in the subscribe window, you see that you got Hello World.
1121200	1124740	If localhost is too boring for you, then you're in luck.
1125380	1132440	Mosquito.org has set up a test broker where you have ports for each type of broker service.
1132540	1137960	You can subscribe to all topics using the hashtag or pound sign.
1137960	1147240	And then use "-v to show topic names along with the data so you don't just get Hello World, you get CarHackingVillage topic Hello World.
1148040	1155540	You can see this note on their website that says don't publish anything sensitive because anyone can be listening.
1155860	1157780	You probably know where this is going.
1157780	1168260	Here are some screenshots from the other day where I have subscribed to all topics on the unencrypted port 1883 test mosquito.org server.
1168260	1176080	We have what seem to be plain text passwords. We've got the lighting and temperature controls for some office.
1176080	1180260	Probably most concerningly an over-the-air update.
1180480	1185980	And then this last person was just publishing basic C4 encoded images.
1186600	1190940	A lot of the data being published here is probably junk.
1190940	1193540	However, some of it does look important.
1193540	1204180	A lot of times, IoT projects or other projects that use MQTT will use a default value of test.mosquito.org as their broker
1204180	1207980	so that the whole project will just work right out of the box.
1208060	1212180	There's usually a note to change it, but of course not everyone does that.
1212180	1221980	I have seen a production TCU unit that had test.mosquito.org commented out in the configuration file, so it can happen.
1222440	1225740	Since this is DEF CON, I should probably talk about security.
1225740	1238640	You likely noticed that in the earlier examples, including test.mosquito.org, there is a pretty minimal set of information that we needed in order to subscribe to a topic or publish information to that topic.
1239120	1252760	We would hope that MQTT offered some kind of security features so we aren't completely wide open to anyone who can guess our IP address of the broker and then sees port 1883 open on their nmap scan.
1253360	1260220	As with the Mosquito test server listings, there are different levels of security that you can set on your broker.
1260300	1272780	For the lowest level, which is the default, you only need to know the host name, port, which default is 1883, a topic, including wildcards, in order to subscribe.
1272860	1279860	Technically, you also need a client ID, but the Mosquito command line tools will make one for you if you don't have one.
1280640	1287600	By default, MQTT does not require authentication or encryption. You can set that up on your own.
1287600	1298720	You can also set up authentication without encryption, but as shown in this screenshot, that means that plain text credentials are visible if you can sniff traffic.
1304360	1311400	Hopefully, that isn't enough security for your connected car architecture and you want to do a little bit more.
1311460	1315240	The next thing to do is to add encryption.
1315480	1318840	We can do MQTT over TLS.
1318840	1327700	We use a certificate to encrypt traffic between the client and the server, which means no more plain text credentials.
1328220	1330960	Random people can't connect to your server.
1330960	1336680	If you try and connect without a certificate, the server won't even respond with an acknowledgement.
1337680	1344100	Your data is going to be encrypted in transit, which is a significant step up in security.
1344100	1347360	I don't think you'll see many car networks with unencrypted traffic.
1347360	1350000	I don't know that you can say the same for IoT.
1351120	1355860	Let's say you have a car network set up with MQTT over TLS.
1355860	1359040	You still need to use defense in depth.
1359400	1373280	If you were to buy a TCU off of eBay, you get console access, you dump the firmware, and now you have the host name, the port number, what a valid client ID might look like, and then the certificate.
1373280	1377940	You could connect to the broker or server using that information.
1378240	1383080	You might think that's not a big deal. You're just going to get information about that TCU.
1383080	1395240	Depending on how you have set up authentication and access control, you might be able to get information about other cars, which is obviously not good.
1395600	1399640	That means we need to employ some level of authentication.
1401140	1405380	We already said that authentication is not required by default.
1405380	1408840	You can configure your broker to authenticate clients.
1408840	1416820	If you are using some kind of paid-for service like AWS IoT, they probably make this even easier for you.
1417280	1424040	You can pass in a username and password to authenticate devices against a device list.
1424040	1429120	But what would be even better than that is to have individual client certificates.
1429900	1436160	Of course, this makes the manufacturing and provisioning process more difficult and more expensive.
1436280	1446020	But in this scenario, if you were to buy a TCU off of eBay and hack it, and you have set up access control correctly, you can only connect as that device.
1446020	1459080	You can't guess other usernames or passwords, especially if you're doing something that is guessable, like the username is the VIN or the IMEI or something like that.
1460040	1465060	If you have authentication set up, you still need to think about authorization.
1465380	1470300	What kind of data or functionality should a single device have access to?
1470300	1479040	Ideally, cars should only be able to see information about that car, not other cars in the fleet, and not system-wide information.
1479740	1489320	If they aren't using individual client certificates, then the authorization might be based off of either the username or the client ID that you have to provide.
1489320	1490960	You should try both.
1492720	1498340	Brokers can be configured to disconnect clients that are asking for unauthorized topics.
1498340	1501180	I don't know how common this is in the wild.
1501180	1514180	For example, I know that AWS IoT lets you configure all kinds of security access violation rules, but I haven't actually seen anyone use it for production.
1515260	1523980	If you manage to get connected in whatever way, what you should try then is to see if the broker allows wildcards.
1523980	1528160	Can you use hashtag or pound sign to connect to all topics?
1528160	1532120	If not, then you need to have a little bit more nuance.
1532120	1548660	Try connecting to, for example, from earlier, car data, and then either a pound sign after that for everything under car data, or car data, and then a plus sign, and then topic names that you already know about from your car.
1549180	1555880	For example, location, latitude, and see if you can get information that way.
1555880	1565040	Just because they have restricted wildcards at one level doesn't necessarily mean that they have filtered them out correctly at all levels, so try a couple different permutations.
1565640	1573100	You should also try the system level topics to see if you can find information about the connected clients.
1573100	1593000	And then if they have locked down all of the wildcards, which hopefully they're doing, you might be able to guess what the other IDs are and essentially go through the same process, but manually trying out different VINs or IMEIs instead of making use of the wildcard.
1593000	1607480	For example, if the ID is a VIN, you can probably guess other valid VINs or look them up somehow. They might also be using the IMEI or the ICC ID of the telematics control unit.
1607740	1621230	Ideally, they're checking your client name or username or certificate to determine whether you're authorized to view this, but if they had to roll their own access control setup, maybe they've made mistakes somewhere.
1622230	1633910	There are other security features that you could use. For example, the access control violation rules mentioned earlier or a VPN to close the attack surface up even more.
1634150	1639270	I haven't used these tools a whole lot, but I found them while making this talk and I thought I'd include them.
1639270	1658710	MQTT Pwn is a whole framework around MQTT testing. The second one is a fuzzer and then the last one is less for security testing and more on the development side. It has a ton of great links for frameworks and client code and things like that in a bunch of different languages.
1659150	1660990	It's a great resource.
1663070	1674990	So, in summary, MQTT is a PubSub protocol that runs on ports 1883 or 8883, depending on whether you're using TLS or not.
1674990	1685390	You have one broker and multiple clients. In the automotive world, the clients are likely TCUs, but might be other things as well, like mobile apps.
1685410	1690850	Information is separated out by topic hierarchies as defined by each system.
1692290	1699610	MQTT security is important because of privacy concerns. You have car location that could be shared over MQTT.
1699810	1713590	You have car safety, if you can do remote lock or unlock, and then over there, updates, where if you're somehow able to get in the middle of that, you could download malicious firmware onto the device.
1713670	1718230	Hopefully, again, you have defense in depth and you have other checks for that.
1719130	1731130	But MQTT is insecure by default, and it is up to the developers to implement authentication, authorization, encryption, and access control correctly.
1731490	1737750	And lastly, MQTT is something that you can use in car hacking, if you have a car with a telematics component.
1737750	1746390	If you don't have that, MQTT is also very common in IoT, so you might be able to practice some of these things with IoT devices you have around your house.
1750090	1756050	I hope this has been useful. I'll be around in Discord for a little bit afterwards.
1756050	1764410	Feel free to hit me up on Discord, CarHackingVillageJamie, or message me on Twitter if you have any questions or anything that I can help with.
1768780	1773820	All right. Jamie Lightfoot, everyone. Thank you very much for your talk.
1774580	1778680	Jamie, I believe you're also in the channel currently.
1782520	1784520	Good evening.
1785220	1789280	Actually, we don't have any questions coming from the audience.
1789280	1806140	If you do have questions and you want to ask Jamie right now, please go ahead and type them into the CHP 101 support text channel, and Jamie would be happy to answer them now.
1806140	1818540	Or I'm sure Jamie's going to be around for some time, and maybe you can catch her direct chat with her or in one of the channels as well.
1820020	1825520	So, Jamie, thank you very much for the talk today. It was really nice.
1826480	1829280	Thank you, and thank you, everyone, for listening.
1830640	1847470	Great. And once again, contact Jamie if you have any questions. If you want to make any comments or just talk about MQTT, excuse me, then talk with her directly.
1848290	1858890	Jamie, thank you for the presentation again. And actually, this was our last presentation, our last talk of the day.