-
Notifications
You must be signed in to change notification settings - Fork 1
/
docker.html
315 lines (213 loc) · 13.5 KB
/
docker.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
<!DOCTYPE html>
<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>How datacats uses Docker — datacats 1.2.0 documentation</title>
<link rel="stylesheet" href="_static/css/datacats_theme.css" type="text/css" />
<link rel="top" title="datacats 1.2.0 documentation" href="index.html"/>
<link rel="next" title="Command Reference" href="commands.html"/>
<link rel="prev" title="FAQ" href="faq.html"/>
<script src="_static/js/modernizr.min.js"></script>
</head>
<body class="wy-body-for-nav" role="document">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search">
<a href="index.html" class="icon icon-home"> datacats
</a>
<div class="version">
1.2.0
</div>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="search.html" method="get">
<input type="text" name="q" placeholder="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
<div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="quickstart.html">Quickstart</a></li>
<li class="toctree-l1"><a class="reference internal" href="guide.html">User Guide</a></li>
<li class="toctree-l1"><a class="reference internal" href="faq.html">FAQ</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="">How datacats uses Docker</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#images-and-containers">Images and Containers</a></li>
<li class="toctree-l2"><a class="reference internal" href="#throw-away-containers">Throw-away containers</a></li>
<li class="toctree-l2"><a class="reference internal" href="#stateless-development">Stateless development</a></li>
<li class="toctree-l2"><a class="reference internal" href="#devops">DevOps</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="commands.html">Command Reference</a></li>
<li class="toctree-l1"><a class="reference internal" href="changelog.html">Change Log</a></li>
</ul>
</div>
</div>
</nav>
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
<nav class="wy-nav-top" role="navigation" aria-label="top navigation">
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="index.html">datacats</a>
</nav>
<div class="wy-nav-content">
<div class="rst-content">
<div role="navigation" aria-label="breadcrumbs navigation">
<ul class="wy-breadcrumbs">
<li><a href="index.html">Docs</a> »</li>
<li>How datacats uses Docker</li>
<li class="wy-breadcrumbs-aside">
</li>
</ul>
<hr/>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<span id="forkongithub"><a href="https://github.com/datacats/datacats">Fork me on GitHub</a></span>
<div class="section" id="how-datacats-uses-docker">
<span id="docker"></span><h1>How datacats uses Docker<a class="headerlink" href="#how-datacats-uses-docker" title="Permalink to this headline">¶</a></h1>
<div class="section" id="images-and-containers">
<h2>Images and Containers<a class="headerlink" href="#images-and-containers" title="Permalink to this headline">¶</a></h2>
<p>datacats puts CKAN inside of Docker containers. Each datacats CKAN environment
is actually at least 3 different Docker containers. You can see this by running
<code class="docutils literal"><span class="pre">datacats</span> <span class="pre">info</span></code> inside your project directory:</p>
<div class="highlight-python"><div class="highlight"><pre>$ datacats info
Environment name: datapusher
Default port: 5716
Environment dir: /Users/dz/source/dcats-envs/datapusher
Data dir: /Users/dz/.datacats/datapusher
Containers: web postgres solr
Available at: http://boot2docker:5716/
</pre></div>
</div>
<p>You can see we have a <code class="docutils literal"><span class="pre">web</span></code> container, a <code class="docutils literal"><span class="pre">postgres</span></code> container, and a <code class="docutils literal"><span class="pre">solr</span></code>
container. Each is responsible for the corresponding piece of CKAN - the postgres
container is running the database, the solr container is running the solr search
engine, and web is running CKAN itself, connecting to solr and postgres as needed.</p>
<p>From a Docker perspective, you can see these containers by running <code class="docutils literal"><span class="pre">docker</span> <span class="pre">ps</span></code>:</p>
<div class="highlight-python"><div class="highlight"><pre>$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a3a0654c3f0a datacats/web:latest "/scripts/web.sh" 26 hours ago Up 26 hours 0.0.0.0:5716->5000/tcp datacats_web_datapusher
2656f1dd7ecc datacats/search:latest "/usr/share/tomcat6/ 26 hours ago Up 26 hours 8080/tcp datacats_solr_datapusher
12dfb5f7950f datacats/postgres:latest "/docker-entrypoint. 26 hours ago Up 26 hours 5432/tcp datacats_postgres_datapusher
</pre></div>
</div>
<p>You can see that each container is using a corresponding Docker image. An image
is like a template used to run each container.</p>
</div>
<div class="section" id="throw-away-containers">
<h2>Throw-away containers<a class="headerlink" href="#throw-away-containers" title="Permalink to this headline">¶</a></h2>
<p>The most important thing to remember about your datacats containers is that they
are <a class="reference external" href="http://12factor.net/disposability">disposable</a>. You can kill any of these containers without worry, for example:</p>
<div class="highlight-python"><div class="highlight"><pre>$ docker rm -f datacats_web_datapusher
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2656f1dd7ecc datacats/search:latest "/usr/share/tomcat6/ 27 hours ago Up 27 hours 8080/tcp datacats_solr_datapusher
12dfb5f7950f datacats/postgres:latest "/docker-entrypoint. 27 hours ago Up 27 hours 5432/tcp datacats_postgres_datapusher
</pre></div>
</div>
<p>No big deal, let’s reload. This will create a new container to serve our CKAN:</p>
<div class="highlight-python"><div class="highlight"><pre>$ datacats reload
Starting web server at http://boot2docker:5716/...
</pre></div>
</div>
</div>
<div class="section" id="stateless-development">
<h2>Stateless development<a class="headerlink" href="#stateless-development" title="Permalink to this headline">¶</a></h2>
<p>The second important thing to note is that containers are more or less immutable -
stuff inside a container doesn’t change. This is not strictly true. Docker does
allow us to go inside a container and do whatever we want. However, this would
make our containers no longer disposable, since there is now state encapsulated
inside of them we presumably wish to not lose. Because of this, datacats treats
containers as stateless and disposable.</p>
<p>This comes with a couple of important exceptions. Our source files and our
database files. These are things that are obviously not
stateless - both our source and our database change constantly as we develop and
use our CKAN site. Our source files - the project directory - is mounted inside
each web container when that container is started.</p>
<p>Why is all of this important? First reason is maintainability. We have all of our
messy moving parts encapsulated inside two discrete places - our source directory
and our database. Everything else is disposable and we don’t have to worry about it
breaking.</p>
</div>
<div class="section" id="devops">
<h2>DevOps<a class="headerlink" href="#devops" title="Permalink to this headline">¶</a></h2>
<p>Second important reason is deployment. Because all of CKAN is running
inside Docker, everything is in tightly controlled image environments that are immune to changes.
We can deploy these same images to a production server and achieve an extremly high
level of <a class="reference external" href="http://12factor.net/dev-prod-parity">development-production parity</a>.</p>
<p>Note also that each of our containers is responsible for running <a class="reference external" href="http://12factor.net/processes">exactly one process</a>.
postgres container is just running postgres, solr is running solr, and web is running
paster or apache. Each of these services is as simple as can be, and this architecture
also allows us to <a class="reference external" href="http://12factor.net/concurrency">scale out</a> each of these parts independently.</p>
<p>Here is an illustrative example to wrap your mind around this architecture.
Let’s perform a database migration:</p>
<div class="highlight-python"><div class="highlight"><pre>datacats paster --plugin=ckan db upgrade
</pre></div>
</div>
<p>In the few seconds that this is running, run the following command:</p>
<div class="highlight-python"><div class="highlight"><pre>$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
56392375f4e9 datacats/web:latest "/scripts/shell.sh - 1 seconds ago Up 1 seconds 5000/tcp grave_mayer
1e6b951d172b datacats/web:latest "/scripts/web.sh" 46 minutes ago Up 46 minutes 0.0.0.0:5716->5000/tcp datacats_web_datapusher
2656f1dd7ecc datacats/search:latest "/usr/share/tomcat6/ 27 hours ago Up 27 hours 8080/tcp datacats_solr_datapusher
12dfb5f7950f datacats/postgres:latest "/docker-entrypoint. 27 hours ago Up 27 hours 5432/tcp datacats_postgres_datapusher
</pre></div>
</div>
<p>We see there are two containers running from the image template <code class="docutils literal"><span class="pre">datacats/web</span></code>.
One of those containers is our database migration. After the migration is done,
<a class="reference external" href="http://12factor.net/admin-processes">the container is destroyed</a>. It may take you some time to get used to this way
of doing things. After all, we just created and destroyed a perfectly good
“virtual machine” just so we can run a 5 second script. How wasteful! But this
is the primary paradigm of how Docker and containers work. The sooner you get
used to it, the happier your experience with Docker will be.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">Traditionaly, we would <code class="docutils literal"><span class="pre">ssh</span></code> into our web server and run the migration scripts.
However, remember that our containers are running only one process each, so they
don’t have an ssh daemon listening which would allow us to <code class="docutils literal"><span class="pre">ssh</span></code> into the server,
even if we wanted to.</p>
</div>
</div>
</div>
</div>
</div>
<footer>
<div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
<a href="commands.html" class="btn btn-neutral float-right" title="Command Reference" accesskey="n">Next <span class="fa fa-arrow-circle-right"></span></a>
<a href="faq.html" class="btn btn-neutral" title="FAQ" accesskey="p"><span class="fa fa-arrow-circle-left"></span> Previous</a>
</div>
<hr/>
<div role="contentinfo">
<p>
© Copyright 2015, boxkite.
</p>
</div>
Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
</div>
</div>
</section>
</div>
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT:'./',
VERSION:'1.2.0',
COLLAPSE_INDEX:false,
FILE_SUFFIX:'.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<script type="text/javascript" src="_static/js/theme.js"></script>
<script type="text/javascript">
jQuery(function () {
SphinxRtdTheme.StickyNav.enable();
});
</script>
</body>
</html>