Skip to content

Commit 68a58fd

Browse files
PresktonPreskton
authored
Refreshing the Infra page b/c it's WAY out of date (#591)
1 parent e7c8f3a commit 68a58fd

File tree

3 files changed

+300
-12
lines changed

3 files changed

+300
-12
lines changed

‎community.hachyderm.io/content/en/docs/infrastructure/_index.md

Lines changed: 300 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -7,26 +7,314 @@ description: >
77
---
88

99
{{% pageinfo %}}
10-
This page is still being built. Check back often.
10+
This page is missing a lot of details - we'll be publishing more in November/December of 2024.
1111
{{% /pageinfo %}}
1212

13-
Hachyderm is taking the first steps towards what we are calling **Open Source Infrastructure** which is where we intend to operate a secure and resilient service completely in the public.
13+
## Footprint
1414

15-
## Hachyderm Today
15+
Today Hachyderm runs a multi-cloud, global topology that is distributed across the following service providers:
1616

17-
Today Hachyderm runs a global topology that is distributed across the following service providers:
17+
- **Edge node "CDN"** - small, lightweight [Linode](https://www.linode.com/) VMs operating around the world. These serve as the front door to Hachyderm and allow us to cache content close to where our users consume it.
18+
- **Mastodon Core** - physical machines operating in [Hetzner](https://www.hetzner.com/) in Falkenstein, Germany.
19+
- **Media Storage** - media (images, videos, audio, etc.) storage hosted in DigitalOcean's [Spaces S3-alike]](https://www.digitalocean.com/products/spaces) service.
1820

19-
- On premise servers in Seattle, Washington also known as "The Watertower".
20-
- Small lightweight [Linode](https://www.linode.com/) VMs operating around the world.
21-
- Core infrastructure operating in [Hetzner](https://www.hetzner.com/) in Falkenstein, Germany.
22-
- Object storage sponsored by [Digital Ocean](https://www.digitalocean.com/products/spaces).
21+
The disparate clouds are linked together using [Tailscale](https://tailscale.com) to create a resilient, secure global virtual private network.
2322

24-
![](topology-v2.png)
23+
![Map of the world with Hachyderm's sites as pins on the map](./infra-2024-11.png)
2524

25+
### Components
2626

27-
### Experimenting in the Public
27+
Key components of our tech stack include:
2828

29-
Hachyderm deeply believes there is untapped value left in computer science. We intend on approaching our infrastructure as an opportunity for safe and thoughtful experiments, similar to how the [International Space Station conducts experiments in orbit](https://www.nasa.gov/mission_pages/station/research/experiments_category). We intend on prototyping new technology, operational models, SRE organizational structure, follow-the-sun patterns, and open source collaborative workstreams for our infrastructure. In the coming months we will be sharing ways in which the broader Hachyderm community can volunteer to support our infrastructure, as well as register hypothesis backed experiments to run with our data and our services.
29+
- [Mastodon](https://github.com/mastodon/mastodon)
30+
- [Arch Linux](https://archlinux.org/) and [Debian](https://www.debian.org/)
31+
- [systemd](https://systemd.io/)
32+
- [Ansible](https://www.ansible.com/)
33+
- [postgresql](https://www.postgresql.org/)
34+
- [ElasticSearch](https://www.elastic.co/elasticsearch)
35+
- [Redis](https://redis.io/)
36+
- [Nginx](https://nginx.org/)
37+
- [Terraform](https://www.terraform.io/)
38+
- Multi-cloud networking via [Tailscale](https://tailscale.com)
39+
- Deployment pipelines that enshrine safety to yield speed and agility
40+
- General cloud engineering and SRE patterns & practices
41+
- Experience with any cloud provider, in particular: [Linode](https://linode.com), [Hetzner](https://hetzner.com), [DigitalOcean](https://digitalocean.com)
3042

31-
Your user data will never be leveraged for an experiment. All user profile data, direct messages, post content, access metrics, demographic detail, and personal information will be restricted from any form of experiment. To be even more direct about, you, as a Hachyderm user will never be leveraged in an experiment. We will not be experimenting on you. We are experimenting on the tools and services that support hachyderm's public services such as prototyping databases, HTTP(s) servers, and compute runtimes.
43+
(Notice we **didn't** say [Kubernetes](https://kubernetes.io) -- we don't currently use it, nor plan to!)
3244

45+
## Experimenting in Public
46+
47+
Hachyderm deeply believes there is untapped value left in computer science. We intend on approaching our infrastructure as an opportunity for safe and thoughtful experiments, similar to how the [International Space Station conducts experiments in orbit](https://www.nasa.gov/mission_pages/station/research/experiments_category).
48+
49+
We intend on prototyping new technology, operational models, SRE organizational structure, follow-the-sun patterns, and open source collaborative workstreams for our infrastructure. In the coming months, we will be sharing ways in which the broader Hachyderm community can volunteer to support our infrastructure, as well as register hypothesis backed experiments to run with our data and our services. We are experimenting on the tools and services that support Hachyderm's services such as prototyping databases, HTTP(s) servers, and compute runtimes.
50+
51+
### ... but *never* for AI
52+
53+
The Hachyderm team will **never** use your user data in an experiment, including for the purposes of training LLMs/AI models.
54+
55+
To be even more direct: you, as a Hachyderm user will never be leveraged in an experiment. We will not be experimenting on you. All user profile data, direct messages, post content, access metrics, demographic detail, and personal information will never be used in any form of experiment.
56+
57+
## Volunteering
58+
59+
If you're interested in volunteering, read more [here](https://github.com/hachyderm/community/discussions/590).
60+
61+
## Infrastructure Placement & Supporting Tools
62+
63+
<table>
64+
<tr>
65+
<td>
66+
<strong>Component</strong>
67+
</td>
68+
<td><strong>Asset Location(s)</strong>
69+
</td>
70+
<td><strong>Provider(s) & Country</strong>
71+
</td>
72+
<td><strong>Services Provided</strong>
73+
</td>
74+
</tr>
75+
<tr>
76+
<td>Edge CDN
77+
</td>
78+
<td>
79+
<ul>
80+
81+
<li>US - California and New Jersey</li>
82+
83+
<li>DE - Frankfurt</li>
84+
85+
<li>JP - Tokyo</li>
86+
87+
<li>(further expansion planned soon)</li>
88+
</ul>
89+
</td>
90+
<td>US - Akamai
91+
</td>
92+
<td>
93+
<ul>
94+
95+
<li>Virtual Machines</li>
96+
97+
<li>Network Backbone</li>
98+
</ul>
99+
</td>
100+
</tr>
101+
<tr>
102+
<td>Mastodon Web
103+
</td>
104+
<td>
105+
<ul>
106+
107+
<li>DE - Frankfurt</li>
108+
</ul>
109+
</td>
110+
<td>DE - Hetzner
111+
</td>
112+
<td>
113+
<ul>
114+
115+
<li>Virtual Machines</li>
116+
117+
<li>Network Backbone</li>
118+
</ul>
119+
</td>
120+
</tr>
121+
<tr>
122+
<td>Mastodon Sidekiq (Queues)
123+
</td>
124+
<td>
125+
<ul>
126+
127+
<li>DE - Frankfurt</li>
128+
</ul>
129+
</td>
130+
<td>DE - Hetzner
131+
</td>
132+
<td>
133+
<ul>
134+
135+
<li>Virtual Machines</li>
136+
137+
<li>Network Backbone</li>
138+
</ul>
139+
</td>
140+
</tr>
141+
<tr>
142+
<td>Mastodon Redis
143+
</td>
144+
<td>
145+
<ul>
146+
147+
<li>DE - Frankfurt</li>
148+
</ul>
149+
</td>
150+
<td>DE - Hetzner
151+
</td>
152+
<td>
153+
<ul>
154+
155+
<li>Virtual Machines</li>
156+
157+
<li>Network Backbone</li>
158+
</ul>
159+
</td>
160+
</tr>
161+
<tr>
162+
<td>Mastodon Streaming
163+
</td>
164+
<td>
165+
<ul>
166+
167+
<li>DE - Frankfurt</li>
168+
</ul>
169+
</td>
170+
<td>DE - Hetzner
171+
</td>
172+
<td>
173+
<ul>
174+
175+
<li>Virtual Machines</li>
176+
177+
<li>Network Backbone</li>
178+
</ul>
179+
</td>
180+
</tr>
181+
<tr>
182+
<td>Mastodon Postgresql (Database)
183+
</td>
184+
<td>
185+
<ul>
186+
187+
<li>DE - Frankfurt</li>
188+
</ul>
189+
</td>
190+
<td>DE - Hetzner
191+
</td>
192+
<td>
193+
<ul>
194+
195+
<li>Virtual Machines</li>
196+
197+
<li>Network Backbone</li>
198+
</ul>
199+
</td>
200+
</tr>
201+
<tr>
202+
<td>Observability Metrics & System Logs
203+
</td>
204+
<td>
205+
<ul>
206+
207+
<li>DE - Frankfurt</li>
208+
</ul>
209+
</td>
210+
<td>DE - Hetzner
211+
</td>
212+
<td>
213+
<ul>
214+
215+
<li>Virtual Machines</li>
216+
217+
<li>Network Backbone</li>
218+
</ul>
219+
</td>
220+
</tr>
221+
<tr>
222+
<td>Mastodon Media Storage
223+
</td>
224+
<td>
225+
<ul>
226+
227+
<li>DE - Frankfurt</li>
228+
</ul>
229+
</td>
230+
<td>US - DigitalOcean
231+
</td>
232+
<td>
233+
<ul>
234+
235+
<li>S3-compatible Storage</li>
236+
</ul>
237+
</td>
238+
</tr>
239+
<tr>
240+
<td>Observability Traces
241+
</td>
242+
<td>
243+
<ul>
244+
245+
<li>US</li>
246+
</ul>
247+
</td>
248+
<td>US - Honeycomb
249+
</td>
250+
<td>
251+
<ul>
252+
253+
<li>Honeycomb OTel Ingestion</li>
254+
</ul>
255+
</td>
256+
</tr>
257+
<tr>
258+
<td>Connectivity Mesh
259+
</td>
260+
<td>
261+
<ul>
262+
263+
<li>Various - present on all Hachyderm VMs</li>
264+
</ul>
265+
</td>
266+
<td>CA - Tailscale
267+
</td>
268+
<td>
269+
<ul>
270+
271+
<li>Virtual Encrypted Network</li>
272+
273+
<li>Machine Identity & Tagging</li>
274+
</ul>
275+
</td>
276+
</tr>
277+
<tr>
278+
<td>Code Version Control
279+
</td>
280+
<td>
281+
<ul>
282+
283+
<li>US - Washington + Virginia</li>
284+
</ul>
285+
</td>
286+
<td>US - Github/Microsoft
287+
</td>
288+
<td>
289+
<ul>
290+
291+
<li>Git Source Control</li>
292+
</ul>
293+
</td>
294+
</tr>
295+
<tr>
296+
<td>Secrets Management
297+
</td>
298+
<td>
299+
<ul>
300+
301+
<li>US - North Virginia</li>
302+
</ul>
303+
</td>
304+
<td>CA - 1Password
305+
</td>
306+
<td>
307+
<ul>
308+
309+
<li>1Password Vault</li>
310+
</ul>
311+
</td>
312+
</tr>
313+
</table>
314+
315+
316+
## Historical Archive
317+
318+
Here's our "v2" infrastructure diagram from circa November 2022:
319+
320+
![](infra-2022-11.png)

‎community.hachyderm.io/content/en/docs/infrastructure/topology-v2.png renamed to ‎community.hachyderm.io/content/en/docs/infrastructure/infra-2022-11.png

File renamed without changes.

‎community.hachyderm.io/content/en/docs/infrastructure/infra-2024-11.png

533 KB
Loading

0 commit comments

Comments
 (0)