1
00:00:00,000 --> 00:00:01,400
Sounds like we're live.

2
00:00:01,400 --> 00:00:12,660
I wanted to start off today's presentation by thanking everybody in the DEF CON community for welcoming me to present on the Storage Network in Tardigrade today.

3
00:00:12,660 --> 00:00:16,960
This is my second time presenting at DEF CON and I'm really excited to be here.

4
00:00:17,000 --> 00:00:31,160
Today what I'm going to do is I'm going to first give a quick update around the Storage Network, which launched in March, talk about the recent wins from an adoption standpoint, what tools are being built on the network, who are we working with.

5
00:00:31,160 --> 00:00:45,600
Then I'm going to go into a quick workshop that will walk you through the process of backing up Kubernetes configuration and object data to the decentralized cloud using an open source tool called Valera, which is sponsored by VMware.

6
00:00:46,280 --> 00:00:49,560
And so with that, let's go ahead and get started.

7
00:00:49,560 --> 00:00:54,940
And if anyone has questions, just post them in the chat and we can get to them at the end right before the workshop.

8
00:00:55,640 --> 00:01:11,560
So as a company, our goal is to build the world's largest, most secure, resilient, performant, and economical cloud storage service without owning or operating a single data center.

9
00:01:11,560 --> 00:01:23,740
And what that really means is that rather than having the cloud controlled by gatekeepers across a small number of data centers, we enable anybody to be the cloud by monetizing their excess hardware capacity, their extra storage space, and their bandwidth.

10
00:01:24,200 --> 00:01:44,760
And so we formed a decentralized cloud storage network that is highly competitive with, say, an Amazon S3 or Azure Blob or Google Object Store just by incentivizing mass amounts of individuals, data centers, crypto enthusiasts to offer their excess capacity to help build the network.

11
00:01:44,760 --> 00:01:58,620
And so in terms of some stats, right, the original prototype for storage was first built in 2014 by Shawn Wilkinson at a hackathon at Texas Bitcoin Hackathon.

12
00:01:58,820 --> 00:02:01,300
It's gone through a lot of iterations since then.

13
00:02:01,840 --> 00:02:15,140
The key milestone was probably in 2019, we began work on version three of the network after hiring Ben Golub, who is the former CEO of Docker, you know, having built the largest open source community by Linux.

14
00:02:15,260 --> 00:02:16,760
And so we launched in March.

15
00:02:17,020 --> 00:02:21,400
And in terms of stats, we're at 26 petabytes of capacity.

16
00:02:21,640 --> 00:02:29,500
We've done a number of cool things, you know, outside of our early customers, we've actually backed up GitHub through a tool called Git Backup.

17
00:02:29,980 --> 00:02:34,020
And that's a pretty cool open repository of data.

18
00:02:34,020 --> 00:02:37,060
And we're helping other people share open data sets as well.

19
00:02:37,060 --> 00:02:49,580
We have about 8,000 storage nodes, more than 8,000 target users, which are developers that are storing data on the platform, and actually 72,000 unique wallet addresses of people that are participating in our token ecosystem.

20
00:02:49,600 --> 00:02:53,560
We've today stored 50 million files across four satellites.

21
00:02:54,540 --> 00:03:07,620
And, you know, are really proud to have been the number two project in terms of development activity in the Ethereum ecosystem in 2019, as measured by sentiment insights and consensus media.

22
00:03:07,620 --> 00:03:22,560
We have a really awesome partner ecosystem that's driven by a number of open source projects who have integrated with Tardigrade directly, whether for backup tools, or for, you know, second level kind of graphic user interfaces on the platform.

23
00:03:22,560 --> 00:03:28,280
One of those that's pretty notable, we'll get into kind of deeper case studies in a second would be MongoDB.

24
00:03:28,360 --> 00:03:34,000
We recently launched a native integration into their enterprise tool DevOps manager.

25
00:03:34,540 --> 00:03:36,620
So let's get into the fun part.

26
00:03:36,620 --> 00:03:37,740
How does Tardigrade work?

27
00:03:37,740 --> 00:03:39,180
What's the security model?

28
00:03:39,180 --> 00:03:44,240
Why is this a better approach to storing data than on traditional cloud data centers?

29
00:03:44,540 --> 00:03:52,500
So the way that Tardigrade works is that there are three peer classes, or really three pieces of software that anybody can run that powers the network.

30
00:03:52,860 --> 00:03:54,520
So first off, there's the uplink.

31
00:03:54,520 --> 00:04:00,140
The uplink uploads data to the network, encrypting it, splitting it into pieces, and spreading it across nodes.

32
00:04:00,140 --> 00:04:06,460
The satellite talks to storage nodes, which are sharing their capacity and monetizing it.

33
00:04:06,460 --> 00:04:20,840
The satellite builds reputation around the storage nodes based off of their uptime, their response time, and their statistical and correlation to each other, and then uses that reputational profile to determine where data gets pushed from the uplink.

34
00:04:20,840 --> 00:04:24,120
And so the satellite acts as kind of the the key decision maker.

35
00:04:24,120 --> 00:04:26,100
It's an open source peer class that anybody can run.

36
00:04:26,100 --> 00:04:35,860
We run a collection of them with industry standard service level agreements through Tardigrade, but anyone can run them and they decide basically where the data gets stored.

37
00:04:35,860 --> 00:04:42,220
So when you upload the data, it gets broken up into pieces and uploaded in parallel directly to storage nodes without touching the satellite.

38
00:04:42,220 --> 00:04:45,000
The satellite's just telling the uplink where to put the data.

39
00:04:45,760 --> 00:04:51,380
So let's talk about the journey of a file, what happens when you upload a file to the decentralized cloud.

40
00:04:51,380 --> 00:04:54,460
It's actually pretty exciting.

41
00:04:54,640 --> 00:04:57,480
So what first happens is the file gets encrypted.

42
00:04:57,540 --> 00:04:58,760
We'll get into the standards in a bit.

43
00:04:58,760 --> 00:05:00,900
It's AES-GCM256.

44
00:05:01,140 --> 00:05:11,820
It then gets broken up into erasure-coded pieces, and we'll get into what that means in just a second as well, and then pushed across statistically uncorrelated endpoints across the world.

45
00:05:12,120 --> 00:05:30,020
And so what's really critical about Tardigrade is not only do you have default encryption and default geo-redundancy, right, features that you might pay extra for with alternative cloud storage providers, but you also have statistical and correlation from an ownership of the hardware standpoint and network infrastructure.

46
00:05:30,020 --> 00:05:37,620
And so your data, you know, there's not one single provider that could pull the plug on your data, making for a much more resilient cloud storage system.

47
00:05:37,960 --> 00:05:41,980
We'll get into kind of how audit and uptime is ensured as well in a second.

48
00:05:42,100 --> 00:05:57,920
So in terms of the security differentiators of Tardigrade, there are a number of really interesting pieces that make Tardigrade more secure, more reliant, more reliable than traditional cloud service providers.

49
00:05:57,920 --> 00:05:59,680
So first is client-side encryption.

50
00:05:59,680 --> 00:06:02,160
So by default, all data is encrypted client-side.

51
00:06:02,160 --> 00:06:12,040
This prevents scenarios like the leaky bucket scenario that Capital One faced, where an exposed S3 bucket was then downloaded by everyone and exposed a bunch of customer data.

52
00:06:12,040 --> 00:06:14,980
So data is encrypted client-side rather than server-side.

53
00:06:14,980 --> 00:06:17,100
It's encrypted at the edge, at the source.

54
00:06:17,620 --> 00:06:28,780
For API keys, we embed logic inside of the API keys using a embedded logic structure called macaroons, which was originally formulated by Google.

55
00:06:28,940 --> 00:06:37,600
What this means is that instead of, you know, querying an access control list, the API key is actually the access control credential.

56
00:06:37,600 --> 00:06:51,080
And so inside the API key, there's embedded logic that can restrict usage of a file from, say, read-write to only read time-bound files to only be read for a certain period of time and is extensible.

57
00:06:51,080 --> 00:06:59,440
And so you can actually embed third-party logic inside of it that enables you to do things like require OAuth verification and pass an OAuth or SAML token.

58
00:06:59,440 --> 00:07:02,920
So there's some really cool constructs with macaroons that we're playing around with.

59
00:07:03,360 --> 00:07:08,160
And today out of the box, we support kind of standard file restriction capabilities.

60
00:07:08,720 --> 00:07:12,220
And all the developer tools on Tardigrade are simple.

61
00:07:12,280 --> 00:07:20,520
And so it's a really easy, sweet software called libuplink that enables you to connect your applications, object storage to the decentralized cloud.

62
00:07:21,200 --> 00:07:22,660
So let's start with encryption.

63
00:07:22,660 --> 00:07:24,440
Encryption is really important.

64
00:07:29,080 --> 00:07:38,000
Encryption is privacy by default on Tardigrade, and if it was only ratio-coded, it'd be easier to take someone else's file.

65
00:07:38,000 --> 00:07:43,920
So in terms of specifications, all encryption, again, occurs at the edge, at the absolute edge of the client.

66
00:07:44,300 --> 00:07:56,280
Using the AES-GCM-256 standard, which is a military-grade standard, the encryption, if you went into libuplink library and forked it, it's open source, you could plug in your own encryption if you wanted to as well.

67
00:07:56,400 --> 00:08:01,100
It's hierarchically deterministic and it's path-based as well.

68
00:08:01,280 --> 00:08:12,580
And so you might have heard of hierarchical deterministic encryption from BIP32, which is the Bitcoin Improvement Protocol 32, which added hierarchical deterministic key derivation for Bitcoin.

69
00:08:12,580 --> 00:08:16,420
So you can have one master wallet and sub-wallets.

70
00:08:16,420 --> 00:08:20,260
On storage, encryption for files works much the same way.

71
00:08:20,280 --> 00:08:25,800
And similar to how on Ethereum or on Bitcoin, your keys are your money.

72
00:08:25,800 --> 00:08:29,140
On Tardigrade, your keys in many ways are your data.

73
00:08:29,140 --> 00:08:37,900
And so protecting that key is really crucial to the security of the network, and that can be held offline as well.

74
00:08:38,240 --> 00:08:43,480
So erasure coding is kind of the next really important differentiator that we have from a security standpoint.

75
00:08:43,700 --> 00:08:48,720
What that means is after you encrypt a file, it's broken up into segments.

76
00:08:48,780 --> 00:08:56,520
Each segment is broken up into 80 pieces of which you only need today 29 pieces of the 80 to reconstitute the file.

77
00:08:56,620 --> 00:09:11,960
And so from a resilience or security standpoint, what that means is that you need 51 uncorrelated nodes with different owners and different geos with different internet service providers to all go offline to put availability at risk at a given point in time.

78
00:09:11,960 --> 00:09:18,940
If nodes are constantly measured for uptime, if any one of them goes down, that data is repaired and new parity pieces are created.

79
00:09:19,220 --> 00:09:23,700
Erasure coding is also one of the core reasons why we have a performance advantage.

80
00:09:23,700 --> 00:09:30,540
So the way that Tardigrade works is instead of a single thread stream, data is streamed in parallel, somewhat like BitTorrent.

81
00:09:30,540 --> 00:09:42,540
But erasure coding adds an extra benefit because when you call 80 pieces and you only need the first 29 responders to reconstitute the file, you're always going to get the fastest 29 pieces.

82
00:09:42,540 --> 00:09:51,680
Any what we call distributed systems, any long tail responses are basically smoothed out of the distribution due to this cool erasure coding schema.

83
00:09:51,680 --> 00:10:00,660
And so this drives, again, both resilience of the data and performance through parallel packet streaming and kind of native concurrency methods in Go.

84
00:10:01,260 --> 00:10:06,360
The third piece, which we chatted about a second ago, is the access management piece that's built around macaroons.

85
00:10:06,380 --> 00:10:15,920
So traditional cloud storage providers use an access control list or an ACL that's stored in a cloud database to control who has permissions to each file.

86
00:10:15,920 --> 00:10:30,480
We've seen a number of devastating attacks with the ACL model, mainly around super user privilege escalation attacks, where someone climbs up the ACL privileges and then changes everyone else's privilege and can create ransomware out of attacks like that.

87
00:10:30,480 --> 00:10:32,720
Tardigrade is natively designed to prevent this.

88
00:10:32,720 --> 00:10:39,120
All access control happens at the edge and is associated with the macro and with the API key.

89
00:10:39,120 --> 00:10:44,760
And so you can actually, again, embed logic into the API key, which determines identity and access management.

90
00:10:44,760 --> 00:11:01,100
And so we think that edge-based identity access management alone is a huge security advantage over traditional ACL cloud-based models because it is much more resilient to privilege escalation attacks and to a number of other attacks as well.

91
00:11:01,800 --> 00:11:06,480
And so Tardigrade is built by default from the ground up for a trustless environment, right?

92
00:11:07,520 --> 00:11:14,660
The idea is that you can store your data on any endpoint, any uncorrelated to endpoint, and it doesn't matter.

93
00:11:14,660 --> 00:11:29,020
It's built for zero trust, zero knowledge environment, making it from the ground up much more secure than solutions that are built for trusted environments because, as we know, trusted third parties often tend to be security holes.

94
00:11:29,520 --> 00:11:43,920
And so again, we have client-side encryption, we have path-based encryption, all the content and metadata is encrypted before it goes to the satellite, the data storage is distributed so there's not a single point of failure, and all the endpoints are uncorrelated,

95
00:11:43,920 --> 00:11:46,780
and all that ties back to the encryption key ownership.

96
00:11:46,780 --> 00:11:57,420
And so we think that this is a really exciting model to try and disrupt a lot of the weaknesses that we see in standard object storage models.

97
00:11:58,160 --> 00:12:00,200
And so how do you get started with Tardigrade?

98
00:12:00,200 --> 00:12:02,760
How do you upgrade your stack today?

99
00:12:02,760 --> 00:12:07,640
Well, here are a few ways that you can integrate Tardigrade or the storage network into your stack.

100
00:12:07,640 --> 00:12:11,920
One of them that we've seen, one early use case, is around database backups.

101
00:12:11,940 --> 00:12:22,280
And so we have tools today for MySQL, Postgres, InfluxDB, which is a time series database, MongoDB, and other distributed databases like Flurry.

102
00:12:22,280 --> 00:12:34,500
We have native backup tools that enable you to either dump or schedule and dedupe regular backups of your database onto the decentralized cloud.

103
00:12:34,500 --> 00:12:40,260
And a number of companies are really interested in this due to the uncorrelated nature of Tardigrade.

104
00:12:40,260 --> 00:13:00,020
So data is stored across 80 independent operators, there's no single point of failure from the storage side, and we think that this creates a better security play for backups and for disaster recovery than storing your data on a centralized provider that's going to be correlated with all the other providers.

105
00:13:01,320 --> 00:13:08,720
In terms of, you know, the MongoDB integration, we're natively integrated into MongoDB's DevOps Manager tool.

106
00:13:08,760 --> 00:13:10,440
And so that's something we're really excited about.

107
00:13:10,440 --> 00:13:13,260
We launched that integration a couple of months ago.

108
00:13:13,260 --> 00:13:30,540
And if you go to MongoDB's website, if you search MongoDB Tardigrade, there's a great blog post that walks you through the process of basically scheduling backups from DevOps Manager to Tardigrade so that your backups are stored and secured on the distributed cloud.

109
00:13:31,040 --> 00:13:35,580
Another great integration that we recently launched was the integration with rClone.

110
00:13:35,700 --> 00:13:40,280
So rClone is a really popular, common command line program to sync files and directories.

111
00:13:40,280 --> 00:13:43,640
It's often referred to as rSync for the cloud.

112
00:13:43,640 --> 00:13:47,680
And so any sysadmin here that's on the call is going to be familiar with rSync.

113
00:13:48,080 --> 00:14:00,300
And rClone is a similar tool that enables you to kind of map file systems, you know, basically map commands, whether they're dump commands or any kind of commands to natively integrate with Tardigrade.

114
00:14:00,300 --> 00:14:02,800
And we have, again, a native integration with rClone.

115
00:14:02,800 --> 00:14:13,340
So if you download the rClone command line suite and go to their website, you'll find that you can just set up Tardigrade just by inputting your API key or your access directly into rClone.

116
00:14:14,000 --> 00:14:25,600
Another interesting integration for users that might not be technical that we've seen a lot of managed service providers and just non-technical end users use is our integration with Duplicati.

117
00:14:25,800 --> 00:14:28,560
So Duplicati, again, is a GUI-based backup tool.

118
00:14:29,120 --> 00:14:33,780
So basically you click on your file system and you choose what you want to back up and it backs it all up.

119
00:14:34,060 --> 00:14:36,660
And so it's great for, like, just native desktop backups.

120
00:14:37,040 --> 00:14:50,280
And it has a lot of cool features around deduplication and compression, which limit, you know, the actual amount of raw data that gets uploaded, which is more economical than a non-compressed, non-deduped backup.

121
00:14:51,300 --> 00:14:59,740
Another really cool thing, which I'll actually show as I stream the workshop, is that, you know, Tardigrade natively supports video streaming today.

122
00:14:59,940 --> 00:15:04,200
And so from the command line, you can generate a link that allows you to stream video.

123
00:15:04,460 --> 00:15:06,780
It's called Tardigrade Share.

124
00:15:06,820 --> 00:15:11,940
And so you can generate an access that's embedded inside of a URL that embeds that macro and logic.

125
00:15:11,940 --> 00:15:14,160
You can see here it's a rather long URL string.

126
00:15:14,160 --> 00:15:16,920
So you might need to use a shortener if you're going to share it publicly.

127
00:15:16,920 --> 00:15:23,340
But it embeds that logic inside of the URL and lets people watch videos and streams it in parallel from the decentralized web.

128
00:15:23,340 --> 00:15:26,840
This is something that's really cool and I think really powerful as well.

129
00:15:26,840 --> 00:15:40,020
I'm really excited to see where we can start to do things like, you know, map link sharing outputs to decentralized DNS systems, like Unstoppable Domains, Handshake, and some of the other ones that have been mentioned earlier today.

130
00:15:40,020 --> 00:15:45,840
I think, you know, decentralized storage with decentralized DNS is going to be something that's pretty cool.

131
00:15:47,140 --> 00:15:52,020
We also have great native integrations with a number of hardware providers.

132
00:15:52,060 --> 00:15:53,740
And so QNAP, you may have heard of them.

133
00:15:53,740 --> 00:15:59,200
They're one of the larger hardware manufacturers out of Taiwan, which there are many.

134
00:15:59,440 --> 00:16:12,840
QNAP creates network-attached storage arrays and, you know, you can back those up directly to Tardigrade with a one-click app that also shares any of the excess capacity to storage, monetizing both your storage and your bandwidth.

135
00:16:12,900 --> 00:16:28,480
And so if you have a QNAP, a Synology, or Western Digital network-attached storage array, we have documentation that shows how to natively and really easily install an application that both backs up your device to the decentralized cloud and then shares excess capacity to help pay for it.

136
00:16:28,640 --> 00:16:31,540
And, you know, that's also, I think, a super cool use case.

137
00:16:32,240 --> 00:16:39,740
Finally, we have a great integration, you know, with Kubernetes, and we'll get into kind of a workshop around this in just a second.

138
00:16:39,740 --> 00:16:52,040
And so we have a native integration with Valero, which is, again, a VMware-sponsored open-source tool that enables you to back up your Kubernetes configuration object data as well as persistent volumes.

139
00:16:52,080 --> 00:16:54,320
And so we'll get into why you might do that, right?

140
00:16:54,320 --> 00:17:06,560
You know, as a DevOps manager, you know, for looking at disaster recovery, disaster prevention, it makes sense to have, you know, backup of your cluster on the distributed cloud where it's globally distributed.

141
00:17:06,560 --> 00:17:14,440
What that means is that it's really easy to rehydrate your cluster to any compute environment rapidly across the globe with a single backup origin.

142
00:17:14,580 --> 00:17:21,160
And so we solve that problem that a lot of developers face around data locality, which is, you know, where do I store my backups?

143
00:17:21,160 --> 00:17:22,260
Where do I store my data?

144
00:17:22,560 --> 00:17:23,420
Do I store it in U.S.

145
00:17:23,420 --> 00:17:23,700
East?

146
00:17:23,700 --> 00:17:24,380
Do I store it in U.S.

147
00:17:24,380 --> 00:17:24,940
West?

148
00:17:24,940 --> 00:17:28,040
Do I replicate it to Asia and Europe?

149
00:17:28,200 --> 00:17:37,940
With Tardigrade, data is replicated globally by default, and it's globally available by default, which makes it great for disaster recovery when you need to rehydrate, you know, it could be anywhere, right?

150
00:17:37,940 --> 00:17:44,740
Maybe you need to rehydrate to Asia, or maybe you need to rehydrate somewhere in the United States with Tardigrade, you know, one backup solves kind of all those use cases.

151
00:17:45,200 --> 00:17:51,540
And so again, right, this is a great platform that was built by developers, really for developers, and really easy to use.

152
00:17:52,140 --> 00:18:01,500
All of the kind of the complexity around erasure coding, encryption, audit, repair, and those kind of backend network functions are all abstracted.

153
00:18:01,500 --> 00:18:16,660
And so as a developer, you're just looking at kind of a basic, you know, CRUD library, where you can create buckets, put objects in buckets, stream them, delete them, very similar to, say, an S3, but at half the price of S3, you know, with better security,

154
00:18:16,660 --> 00:18:20,760
resilience, and often better performance as well, depending on the use case.

155
00:18:21,580 --> 00:18:32,060
And so again, super easy user interface, I won't get into how to generate a key for this next demo, we're just gonna assume that that's already happened, where you can create an account and generate an API key in just a couple of clicks.

156
00:18:32,260 --> 00:18:34,900
And if you're interested, I recommend you do so.

157
00:18:35,100 --> 00:18:37,640
We also drop in S3 compatibility.

158
00:18:37,880 --> 00:18:47,820
And so if you, and this is really a great demo that I often show, you can actually reconfigure the AWS command line to talk directly to the decentralized cloud through a gateway hosted process.

159
00:18:47,820 --> 00:18:52,800
And so this is a fork of Medio, which provides S3 compatibility for Tardigrade.

160
00:18:52,820 --> 00:19:07,200
And so if your application today is using Amazon S3, and you want to migrate to the decentralized cloud to cut your cloud storage bill in half, and have better resilience and performance, you can do that by running the gateway and easily migrate over without having to rewrite any code,

161
00:19:07,200 --> 00:19:09,800
which is also really powerful.

162
00:19:10,920 --> 00:19:14,140
And so that's the quick overview.

163
00:19:14,760 --> 00:19:18,340
Ajit, does it make sense to answer questions now before I get into the hands-on workshop?

164
00:19:18,340 --> 00:19:20,960
Or should we wait till the end for questions?

165
00:19:30,380 --> 00:19:31,340
Cool.

166
00:19:33,840 --> 00:19:35,240
Anyone have any questions?

167
00:19:42,340 --> 00:19:43,980
Well, we'll go ahead and get started.

168
00:19:43,980 --> 00:19:45,680
And if you have questions, post them in the chat.

169
00:19:45,680 --> 00:19:56,780
So for the workshop today, we're going to walk through the process of creating a decentralized storage backend for Kubernetes that uses Tardigrade and Valero toolset to make that possible.

170
00:19:57,620 --> 00:20:01,140
And so let's go a little bit deeper in what Valero is.

171
00:20:01,140 --> 00:20:05,760
Valero is a tool that enables you to back up and migrate Kubernetes clusters.

172
00:20:05,880 --> 00:20:14,180
And so Valero natively uses the Kubernetes API to capture the state of the cluster resources, which makes it easy to rehydrate at any target.

173
00:20:14,180 --> 00:20:25,100
And so again, Tardigrade is a great backup option for this tool because it's globally distributed, because it's highly economical, and because it's highly secure, encrypted, and resilient as well.

174
00:20:26,700 --> 00:20:38,700
And here's kind of a quick little diagram, by backing up Tardigrade, you can restore your cluster to any compute environment, whether it's cloud or self-hosted globally, really quickly.

175
00:20:39,320 --> 00:20:40,880
It's really easy to install.

176
00:20:40,880 --> 00:20:46,300
There's a simple install command, and then you point it at your existing kubes environment, and you can back it up.

177
00:20:46,440 --> 00:20:49,420
And I'll go ahead and show that off now.

178
00:20:49,600 --> 00:20:59,740
And so what I'll do is I'll stream a demo from Ivan, one of our solution architects and engineers at Storage Labs, and he'll walk through that process.

179
00:20:59,740 --> 00:21:08,920
But first, in terms of early use cases and wins, we've had a number of customers who already started using Valera to backup their Kubernetes databases, sorry, their Kubernetes instances today.

180
00:21:09,320 --> 00:21:15,420
And so what's really great, what we've heard is that backups can capture a subset of the resources, so it's really flexible.

181
00:21:15,420 --> 00:21:18,700
You can filter on namespace, or on label, or on resource type.

182
00:21:18,700 --> 00:21:22,840
So you can really granularly define what gets backed up.

183
00:21:23,780 --> 00:21:29,700
Users also are excited that it backs up the underlying etcd database.

184
00:21:30,840 --> 00:21:36,480
So often, other backup solutions don't backup the etcd database, but Valera does.

185
00:21:36,760 --> 00:21:39,960
And then also, the resources are exposed to the API servers.

186
00:21:40,000 --> 00:21:44,040
And so even if the resources are in separate databases, they can also be backed up.

187
00:21:44,040 --> 00:21:51,880
And so for Kubernetes applications that are looking to improve their disaster prevention strategy, this is one piece of a larger puzzle, right?

188
00:21:51,880 --> 00:21:57,120
You need to also backup all the state associated with your existing databases and persistent volumes.

189
00:21:57,120 --> 00:22:07,200
But this is kind of a really great place to start in tinkering with the decentralized cloud to improve the resiliency of your cluster.

190
00:22:07,600 --> 00:22:14,380
And so what I'm going to do is I'm actually stream this demo directly from the decentralized cloud to show you how performant it is.

191
00:22:14,380 --> 00:22:20,320
If this doesn't work, if the internet gods shame me, I can switch over.

192
00:22:20,320 --> 00:22:21,960
But let's go ahead and get this started.

193
00:22:23,260 --> 00:22:28,020
In this video, I am going to show you how to use the Valera storage plugin.

194
00:22:28,020 --> 00:22:34,760
Valera is a tool that allows you to backup Kubernetes configurations and persistent volume snapshots.

195
00:22:35,360 --> 00:22:41,380
In order to use the storage Valera plugin, you have to create an account in Tably server.

196
00:22:41,820 --> 00:22:50,520
You can go wherever Tably server, sign up, create a project, and after you have a project, create an API key for that project.

197
00:22:50,640 --> 00:23:01,160
Once you have that, make sure that you write down or you copy and paste that API key because it's the only time that you will see that API key and keep it.

198
00:23:01,440 --> 00:23:04,280
So after that, install uplink-cli.

199
00:23:05,580 --> 00:23:11,520
That is the command line tool that Tably has for accessing to the network from the command line.

200
00:23:13,500 --> 00:23:17,100
So with uplink installed, you call setup.

201
00:23:17,440 --> 00:23:32,480
After you select the server where you sign up, in my case it's US Central, select a name or use the default one for the access, grab your API key and set it, and just select a passphrase.

202
00:23:37,820 --> 00:23:44,020
Once you did that, you need to grab the access grant that has been generated with this data.

203
00:23:44,560 --> 00:23:51,620
The access grant is stored in the configuration file because I haven't specified any configuration file.

204
00:23:52,540 --> 00:23:56,040
The uplink has created the configuration file in the default folder.

205
00:23:56,060 --> 00:24:06,020
The default folder is in the case of a unit machine in my home directory under local, share, storage, uplink, config file, config.yaml.

206
00:24:06,060 --> 00:24:08,600
So I want to grab that access.

207
00:24:08,600 --> 00:24:14,520
That is what I need for configuring the developer storage plugin.

208
00:24:15,160 --> 00:24:18,120
I copy the access.

209
00:24:18,120 --> 00:24:25,700
I am going to export to one environment variable that I am going to name it access, store access.

210
00:24:28,860 --> 00:24:34,220
Because I will require that environment variable every time that I need this access.

211
00:24:34,520 --> 00:24:40,460
And I will avoid to pollute this demo in every command with this long string.

212
00:24:42,120 --> 00:24:45,240
Bear in mind that the API key is not something that you have to share.

213
00:24:45,240 --> 00:24:48,940
In my case, I created this API key just for the purpose of this demo.

214
00:24:48,940 --> 00:24:57,000
Right after, I will delete it to avoid that anybody can use it for using, obviously, my network.

215
00:24:57,760 --> 00:25:00,660
So see my files and create files on it.

216
00:25:03,620 --> 00:25:10,800
So the first thing that I am going to do is create a bucket.

217
00:25:10,800 --> 00:25:19,320
This bucket is needed because once you set up that, when you specify a backup in Galero, you have to specify a bucket.

218
00:25:19,620 --> 00:25:23,180
And the bucket has to exist before doing the first backup.

219
00:25:23,240 --> 00:25:28,100
So let's make sure that I have the bucket, uplink.tls.

220
00:25:28,280 --> 00:25:35,040
So right now I have this bucket Galero, but I am going to delete it because I want to start it from scratch.

221
00:25:35,560 --> 00:25:39,440
I remove the bucket with whatever content it has.

222
00:25:42,590 --> 00:25:51,790
Now I don't have any bucket, so I am going to create a bucket that is going to call, I am going to call it Galero.

223
00:25:54,070 --> 00:25:58,010
Once I created a bucket, obviously, that bucket is empty.

224
00:25:58,010 --> 00:25:59,750
But let's show it.

225
00:26:00,110 --> 00:26:00,810
Okay.

226
00:26:02,090 --> 00:26:09,470
Now I am going to install Galero in a Kubernetes cluster that I am running in my log.

227
00:26:09,470 --> 00:26:13,610
At the same time that I am installing Galero, I am defining a bucket location.

228
00:26:14,090 --> 00:26:21,850
The bucket location that I am defining is using the target grade plugin.

229
00:26:23,010 --> 00:26:33,390
Okay, so let's install Galero with provider target grade, that is the name of our plugin,

230
00:26:36,810 --> 00:26:44,930
and specify the image where we store that provider, that is storage third-party

231
00:26:48,430 --> 00:26:49,990
slash Galero.

232
00:26:55,140 --> 00:26:57,660
And I am specifying this tag.

233
00:26:57,660 --> 00:27:00,220
This is a local image that I have in my local only.

234
00:27:00,220 --> 00:27:04,820
This is not available online, but in Docker Hub we have the previous version.

235
00:27:04,820 --> 00:27:10,100
We are still working on this plugin, so we are improving, and I am using this one because it's much better.

236
00:27:10,980 --> 00:27:13,620
After, I have to specify a bucket.

237
00:27:13,760 --> 00:27:33,980
In this case, it's the bucket where I am going to do the backups, that is the Galero bucket that I created right before, and specifically tell to Galero that I am not going to backup as an option, persistent volume as an option.

238
00:27:33,980 --> 00:27:39,740
This is because our plugin doesn't support it yet, but hopefully we will support it in the future.

239
00:27:40,200 --> 00:27:56,800
And after, so I have to, for this configuration location that I am creating, at the same time that I am installing Galero, I have to provide the access grant that is the only configuration parameter that our plugin requires.

240
00:28:02,350 --> 00:28:09,970
Oops, I mistyped the command, but whatever.

241
00:28:11,530 --> 00:28:29,070
Oh yeah, and another thing that I have to do is specify no secret, because Galero requires that you specify a secret, or if you don't need it, if you don't need one, you have to specify that you don't want, that you cannot leave it empty.

242
00:28:29,250 --> 00:28:32,190
So in our case, our plugin doesn't require it.

243
00:28:32,830 --> 00:29:09,870
So now Galero is installed, and I can see that in my cluster, just getting deployment from components components equal Galero from, so from namespace and space Galero, that is components namespace deployment components

244
00:29:15,780 --> 00:29:24,000
and deployments of components of Galero namespace of Galero, that's what, this is component.

245
00:29:24,740 --> 00:29:28,220
Okay, that is what the installation has created.

246
00:29:31,240 --> 00:29:34,890
What I'm going to do for the demo is creating an Nginx service.

247
00:29:35,700 --> 00:29:44,980
And this is the service that we are going to do a backup, we are going to destroy it, like simulating a disaster, and after we'll recover from the backup.

248
00:29:46,640 --> 00:29:53,440
The Nginx service is something, is an example that it comes with the Galero installation.

249
00:29:53,440 --> 00:29:57,800
So Galero has a folder inside of the tar file that has several examples.

250
00:29:58,260 --> 00:30:01,200
This is the folder that I copied from the tar file.

251
00:30:01,200 --> 00:30:03,140
So let's deploy the

252
00:30:06,660 --> 00:30:09,840
the Nginx service.

253
00:30:12,690 --> 00:30:20,310
Once I did that, I can see in my Kubernetes cluster that I have

254
00:30:25,160 --> 00:30:29,100
they have a namespace that is called Nginx example and I have

255
00:30:33,520 --> 00:30:39,080
and I have a service, the Nginx service.

256
00:30:41,100 --> 00:30:43,380
So now let's do a backup.

257
00:30:43,380 --> 00:30:50,880
But before doing that, let's see if backups, Galero backups get.

258
00:30:52,320 --> 00:30:57,460
So this backup get doesn't show anything because we don't have any backup.

259
00:30:57,460 --> 00:30:59,180
So let's create one.

260
00:31:09,980 --> 00:31:11,200
Galero backup

261
00:31:15,360 --> 00:31:18,060
great Nginx backup.

262
00:31:18,060 --> 00:31:32,940
I call this backup Nginx backup and I only want to backup the the resources that have the level up Nginx, not everything.

263
00:31:34,240 --> 00:31:35,620
So I create the backup.

264
00:31:35,620 --> 00:31:39,740
I can do backup get again and my backup is in progress.

265
00:31:40,600 --> 00:31:44,340
And I wait a bit more and after it says that it's complete.

266
00:31:44,900 --> 00:31:49,860
But okay, what we can do now is see what we have in our bucket.

267
00:31:50,620 --> 00:32:00,500
We go away with the uplink ls and just list the, we can list the the Galero bucket.

268
00:32:00,760 --> 00:32:03,300
Now you can see that it's not empty.

269
00:32:03,300 --> 00:32:15,700
There is a directory that is called backups and with, well it's not a directory because we have keys but there is this key inside of the kubernetes but it's kind of simulating a directory.

270
00:32:16,740 --> 00:32:25,240
Nginx backup and we see that under that prefix we have all these files that Galero has created.

271
00:32:25,240 --> 00:32:28,500
So great, we have our backup in our target network.

272
00:32:29,060 --> 00:32:35,140
So what we are going to do now is simulating this this disaster.

273
00:32:36,120 --> 00:32:46,610
For doing that we are going to delete the entire namespace of Nginx, call Nginx example.

274
00:32:49,000 --> 00:32:55,190
Oh, a disaster is happening now and that namespace gets deleted.

275
00:32:57,580 --> 00:33:00,140
We wait until we finish.

276
00:33:06,030 --> 00:33:11,350
And now we are going to see how if we are getting the services

277
00:33:16,380 --> 00:33:19,180
the services from the namespace

278
00:33:24,520 --> 00:33:26,680
Nginx example.

279
00:33:26,900 --> 00:33:29,980
So basically there

280
00:33:33,170 --> 00:33:34,810
is no resources found.

281
00:33:34,950 --> 00:33:47,590
At the same time if we try to get this namespace from the cluster, so basically it's not found.

282
00:33:47,590 --> 00:33:50,930
Okay, that's we are confirming that disaster has happened.

283
00:33:51,670 --> 00:33:53,690
Now we are going to restore.

284
00:33:54,010 --> 00:34:00,710
Galero has a command that is restore and you have to you can say what you are going to restore.

285
00:34:00,710 --> 00:34:08,490
So we are going to restore creating from a backup that is called Nginx backup.

286
00:34:08,990 --> 00:34:11,530
That is the backup what we created previously.

287
00:34:12,590 --> 00:34:14,050
With a restore.

288
00:34:14,050 --> 00:34:20,870
And now Galero restore get says that it's complete.

289
00:34:20,970 --> 00:34:28,590
So if we are going now to list this so that is the namespace it should be there and it's there.

290
00:34:28,590 --> 00:34:32,730
And after we list the service it should also be there and it's there.

291
00:34:32,730 --> 00:34:38,210
So we have recovered our Kubernetes configuration from the delegate network.

292
00:34:38,950 --> 00:34:46,090
Okay, but now what we can do as well is at some point you don't want this backup anymore.

293
00:34:46,090 --> 00:34:47,870
So we can delete the backup.

294
00:34:49,010 --> 00:34:58,790
Sorry, with Galero backup delete and you have to specify the name Nginx backup.

295
00:35:00,910 --> 00:35:11,570
And after it asks for confirmation we confirm and after we do Galero backup get we don't have any backup.

296
00:35:12,510 --> 00:35:19,550
So basically with this we unregister this backup so the backup is not available anymore in Galero.

297
00:35:19,550 --> 00:35:27,710
But obviously we would like to know if those files are stored has been deleted by Galero from our delegate network.

298
00:35:28,970 --> 00:35:30,450
Let's see that.

299
00:35:30,450 --> 00:35:40,490
So for that reason I am going to list again my backup, my Galero backup, and now it's empty.

300
00:35:40,530 --> 00:35:42,860
So basically everything works as expected.

301
00:35:43,170 --> 00:35:55,310
I hope that you enjoyed the demo and I will encourage you to try our Galero plugin with our great delegate object storage decentralized object storage network.

302
00:35:55,550 --> 00:35:57,750
See you next time.

303
00:36:00,000 --> 00:36:02,280
Okay, thank you Ivan.

304
00:36:02,280 --> 00:36:13,320
And again that was streaming directly from the decentralized cloud which I think is pretty neat and pretty telling of some of the capabilities of Tardigrade.

305
00:36:13,720 --> 00:36:23,080
And so with that, that's the conclusion of my presentation and our Valero workshop for backing up Kubernetes data to the decentralized cloud.

306
00:36:23,080 --> 00:36:25,500
Does anyone have any questions?

307
00:36:49,690 --> 00:36:51,830
Looks like I see one popping up in the chat.

308
00:36:54,470 --> 00:36:58,110
If anyone here is a developer, questions in Twitch.

309
00:36:58,110 --> 00:37:02,290
I don't know Ajit if you can rattle those off, I'd be happy to answer them.

310
00:37:02,290 --> 00:37:19,230
But in the meantime, if anyone here is a developer, you can get started with Tardigrade at documentation.tardigrade.io and we have a number of walkthroughs on the website around just basically uploading data, sharing data, streaming it on the internet,

311
00:37:19,230 --> 00:37:24,450
scheduling backups with databases, creating graphic user interfaces around that data.

312
00:37:24,450 --> 00:37:28,690
You can find all of those walkthroughs and more on the documentation for Tardigrade.

313
00:37:42,360 --> 00:37:43,800
No problem.

314
00:37:44,920 --> 00:37:49,040
So with that, that means the presentation went well.

315
00:37:49,040 --> 00:37:50,840
I answered everyone's questions.

316
00:37:51,040 --> 00:37:59,000
And if anyone else has any other questions after the call, you can email me directly kevin at storj storage.io.

317
00:37:59,140 --> 00:38:07,720
And we're always looking for partners and for developers who want to build the next generation of cloud applications on the decentralized cloud.

318
00:38:07,720 --> 00:38:14,420
We're happy to give you the white glove treatment and help you get building or migrate your application to Tardigrade.

319
00:38:14,420 --> 00:38:15,480
So thanks again.

320
00:38:15,480 --> 00:38:18,120
And thanks again for Ajit for hosting me.

321
00:38:18,120 --> 00:38:20,780
And I hope everyone has a great afternoon and happy Friday.

322
00:38:20,780 --> 00:38:21,940
Happy almost weekend.