-
Notifications
You must be signed in to change notification settings - Fork 2
Expand file tree
/
Copy pathlecture4.html
More file actions
448 lines (407 loc) · 16.1 KB
/
lecture4.html
File metadata and controls
448 lines (407 loc) · 16.1 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
<!DOCTYPE html>
<!--
Web 2.0, CTU course slides
(cc) 2010-2014 Tomas Vitvar, tomas@vitvar.com
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta name="course" content="Web 2.0" />
<meta name="lecture" content="Lecture 4" />
<meta name="keywords" content="SaaS, IaaS, PaaS" />
<link type="text/css" rel="stylesheet" href="css/meta.css">
</link>
<link type="text/css" rel="stylesheet" href="css/ctu-fit.css">
</link>
<link type="text/css" rel="stylesheet" href="humla/lib/core/humla.css">
</link>
<script type="text/javascript" src="humla/lib/humla.js"></script>
<title>Containers</title>
</head>
<body>
<footer>
<p><b>#META_LECTURE#: #TITLE#</b>, <span class="meta_semester" />,
<span class="meta_twitter" />
</p>
<p><b>‒ #SLIDE_NO# ‒</b></p>
</footer>
<div class="slide intro">
<hgroup>
<h1><span class="meta_course" /></h1>
<h2>#META_LECTURE#: #TITLE#</h2>
</hgroup>
<div class="author">
<p class="meta_author" />
<p><span class="meta_email" /> • <span class="meta_twitter" /> •
<span class="meta_web" />
</p>
</div>
<center>
<div class="meta_logo"></div>
</center>
<div class="org">
<p class="meta_org" />
<p><span class="meta_orgfac" /> • <span class="meta_field" />
• <span class="meta_orgweb" /></p>
</div>
<div class="etc">
<div class="text-info">
Modified: #LAST_MODIFIED#<br />
Humla v#HUMLA_VERSION#
</div>
<a href="http://creativecommons.org/licenses/by-sa/3.0/">
<div class="license"></div>
</a>
<div class="oppa"></div>
</div>
</div>
<div class="slide outline"></div>
<section>
<header>Overview</header>
<div class="slide">
<hgroup>
<h1>Virtual Machines vs. Containers</h1>
</hgroup>
<div class="h-drawing" style="height:450px; margin-top: 30px" id="13MxlAselSiJfnL7o8NwX87x2SzlPbLdJ56Zi87Ex00w">
</div>
</div>
<div class="slide">
<hgroup>
<h1>Overview</h1>
</hgroup>
<ul class="x-small">
<li>Linux Containers</li>
<ul>
<li>Introduced in 2008</li>
<li>Allow to run a process tree in a isolated system-level "virtualization"</li>
<li>Use much less resources and disk space than traditional virtualization</li>
</ul>
<li>Implementations</li>
<ul>
<li>LXC – default implementation in Linux</li>
<li>Docker Containers</li>
<ul>
<li>Builds on Linux namespaces and union file system (OverlayFS)</li>
<li>A way to build, commit and share images</li>
<li>Build images using a description file called Dockerfile</li>
<li>Large number of available base and re-usable images</li>
</ul>
</ul>
<li>Monolithic design originally</li>
<ul>
<li>Now several layers</li>
<li>container runtime</li>
<li>container engine</li>
</ul>
<div class="h-drawing" id="1XcnPNd8sFL3BztQqVtv6Xf4XUNrpzQHAsDm9Ma6D_S8"
style="text-align:left; height: 160px; margin-top: -120px; margin-left: 350px" />
</ul>
</div>
<div class="slide">
<hgroup>
<h1>Containerd</h1>
</hgroup>
<img src="img/container-arch.jpeg" style="height: 240px; margin-top: 20px; margin-left: 20px" />
<ul class="xx-small">
<li>Container engine</li>
<ul>
<li>Accepts user inputs (via CLI or API), pulling images from registry, preparing metadata to be passed to
container runtime</li>
</ul>
<li>Container runtime</li>
<ul>
<li>Abstraction from syscalls or OS specific functionality to run containers on linux, windows, solaris,
etc.</li>
<li>Uses <code>runc</code> and <code>container-shim</code></li>
<li>Communicates with kernel to start containerized processes</li>
</ul>
</ul>
</div>
<div class="slide">
<hgroup>
<h1>Terminology</h1>
</hgroup>
<ul class="xx-small">
<li>Image</li>
<ul>
<li>An image contains a union of layered filesystems stacked on top of each other</li>
<li>Immutable, it does not have state and it never changes</li>
</ul>
<li>Container</li>
<ul>
<li>One or more processes running in one or more isolated namespaces in a filesystem provided by the image
</li>
</ul>
<li>Container Engine/Runtime</li>
<ul>
<li>The core processes providing container capabilities on a host</li>
</ul>
<li>Client</li>
<ul>
<li>An app (e.g. CLI, custom app), communicates with a container engine by its API</li>
</ul>
<li>Registry</li>
<ul>
<li>A hosted service containing repository of images</li>
<li>A registry provides a registry API to search, pull and push images</li>
<li>Docker Hub is the default Docker registry</li>
</ul>
<li>Swarm</li>
<ul>
<li>A cluster of one or more docker engines</li>
</ul>
</ul>
</div>
</section>
<div class="slide outline"></div>
<section>
<header>Linux Namespaces</header>
<div class="slide">
<hgroup>
<h1>Linux Namespaces</h1>
</hgroup>
<ul class="xx-small">
<li>Isolation of Linux processes, there are <b>7 namespaces</b></li>
<ul>
<li>Mount, UTS, IPC, PID, Network, User, Cgroup</li>
<li>By default, every process is a member of a default namespace of each type</li>
<li>In case no additional namespace configuration is in place, processes and all their direct children will
reside in this exact namespace</li>
<li>Run <code>lsns</code> to check namespaces the process is in</li>
<pre class="brush: plain; gutter: 'false'">
$ lsns
NS TYPE NPROCS PID USER COMMAND
4026531836 pid 2 30873 oracle -bash
4026531837 user 108 1636 oracle /bin/bash /u01/oracle/scripts/startWebLogicContainer.sh
4026531838 uts 2 30873 oracle -bash
4026531839 ipc 2 30873 oracle -bash
4026531840 mnt 2 30873 oracle -bash
4026531956 net 108 1636 oracle /bin/bash /u01/oracle/scripts/startWebLogicContainer.sh
4026532185 mnt 13 13542 oracle /bin/bash /u01/oracle/scripts/startNM_ohs.sh
4026532192 pid 13 2798 oracle /bin/bash /u01/oracle/scripts/startNM_ohs.sh
...
</pre>
</ul>
<li>Flexible configuration, for example:</li>
<ul>
<li>You can run two apps that only share the network namespace, e.g. <code>4026531956</code></li>
<li>The apps can talk to each other</li>
<li>Any other app (not in this namespace) won't be able to talk to the apps</li>
</ul>
</ul>
</div>
<div class="slide">
<hgroup>
<h1>Types: mnt, uts, ipc and pid</h1>
</hgroup>
<ul class="xx-small">
<li><code>mnt</code> namespace</li>
<ul>
<li>Isolates filesystem mount points</li>
<li>Restricts the view of the global file hierarchy</li>
<li>Each namespace has its own set of mount points</li>
</ul>
<li><code>uts</code> namespace</li>
<ul>
<li>The value of the hostname is isolated between different UTS namespaces</li>
</ul>
<li><code>ipc</code> namespace</li>
<ul>
<li>Isolates interprocess communication resources</li>
<li>message queues, semaphore, and shared memory</li>
</ul>
<li><code>pid</code> namespace</li>
<ul>
<li>Isolates PID number space</li>
<li>A process ID number space gets isolated</li>
<ul>
<li>Processes can have PIDs starting from the value 1</li>
<li>Real PIDs outside of the namespace of the same process is a different number</li>
</ul>
<li>Containers have their own init processes with a PID value of 1</li>
</ul>
</ul>
</div>
<div class="slide">
<hgroup>
<h1>Types: net</h1>
</hgroup>
<ul class="xx-small">
<li><code>net</code> namespace</li>
<ul>
<li>Processes have their own private network stack (interfaces, routing tables, sockets)</li>
<li>Communication with external network stack is done by a virtual ethernet bridge</li>
<img src="img/container-bridge.png" style="height: 200px; margin-top: 10px; margin-bottom: 10px" />
<li>On the host there is a <b>userland proxy</b> or <b>NAT</b></li>
<ul>
<li>NAT is a prefered solution over userland proxy (<code>/usr/bin/docker-proxy</code>)</li>
<li>Lack of NAT hairpinning may prevent to use NAT</li>
</ul>
<li>Use case</li>
<ul>
<li>Multiple services binding to the same port on a single machine, e.g. <code>tcp/80</code></li>
<li>A port in the host is mapped to the port exposed by a process in the NS</li>
</ul>
</ul>
</ul>
</div>
<div class="slide">
<hgroup>
<h1>Types: User & Cgroup</h1>
</hgroup>
<ul class="xx-small">
<li><code>user</code> namespace</li>
<ul>
<li>Isolates user and group IDs (UIDs/GIDs) between processes</li>
<li>Allows a process to have different privileges inside and outside the namespace</li>
<li>Enables <b>rootless containers</b> (process is non-root on host, but appears as root inside)</li>
<li>Example</li>
<ul>
<li>A process runs as UID 0 (root) inside the container,<br />but maps to a regular UID on the host</li>
</ul>
</li>
</ul>
<li><code>cgroup</code> namespace</li>
<ul>
<li><b>cgroups</b> (control groups)</li>
<ul>
<li>Kernel feature to limit and measure process resource usage (CPU, memory, I/O)</li>
</ul>
<li>cgroup namespace</li>
<ul>
<li>Isolates the view of the cgroup hierarchy for each process</li>
<li>Prevents a container from seeing/modifying cgroups of the host/other containers</li>
<li>Improves security by restricting what resource controls a container can observe</li>
<li><b>Example:</b> A container only sees its own CPU/memory usage limits, not the host’s full cgroup tree
</li>
</ul>
</ul>
</div>
</section>
<div class="slide outline"></div>
<section>
<header>Container Image</header>
<div class="slide">
<hgroup>
<h1>Container Images</h1>
</hgroup>
<img style="margin-left: 50px; margin-top: 15px; margin-bottom: 10px; height: 280px"
src="./img/docker-container-stack.png" />
<ul class="x-small" style="margin-left: -10px">
<ul>
<li>Containers are made up of R/O layers via a storage driver<br />(OverlayFS, AUFS, etc.)</li>
<li>Containers are designed to support a single application</li>
<li>Instances are ephemeral, persistent data is stored in bind
mounts or data volume containers.</li>
</ul>
</ul>
</div>
<div class="slide">
<hgroup>
<h1>Image Layering with OverlayFS</h1>
</hgroup>
<ul class="x-small">
<li>OverlayFS</li>
<ul>
<li>A filesystem service implementing a <b>union mount</b> for other file systems.</li>
<li>Docker uses <code>overlay</code> and <code>overlay2</code> storage drivers
to build and manage on-disk structures of images and containers.</li>
</ul>
<li>Image Layering</li>
<ul>
<li>OverlayFS takes two directories on a single Linux host, layers one on top of the other, and provides
a single unified view.</li>
<li>Only works for two layers, in multi-layered images hard links are used to
reference data shared with lower layers.</li>
</ul>
<img src="img/overlay_constructs.jpg" style="width: 700px; margin-top: 10px" />
</ul>
</div>
<div class="slide">
<hgroup>
<h1>Image Layers Example</h1>
</hgroup>
<ul class="xx-small" style="margin-top:20px; zoom:0.9">
<li>Pulling out the image from the registry</li>
<pre class="brush: plain; gutter: 'false'">
$ docker pull ubuntu
Using default tag: latest
latest: Pulling from library/ubuntu
5ba4f30e5bea: Pull complete
9d7d19c9dc56: Pull complete
ac6ad7efd0f9: Pull complete
e7491a747824: Pull complete
a3ed95caeb02: Pull complete
Digest: sha256:46fb5d001b88ad904c5c732b086b596b92cfb4a4840a3abd0e35dbb6870585e4
Status: Downloaded newer image for ubuntu:latest
</pre>
<ul>
<li>Each image layer has its own directory under <code>/var/lib/docker/overlay/</code>.</li>
<li>This is where the contents of each image layer are stored.</li>
</ul>
<li>Directories on the file system</li>
<pre class="brush: plain; gutter: 'false'">
$ ls -l /var/lib/docker/overlay/
total 20
drwx------ 3 root root 4096 Jun 20 16:11 38f3ed2eac129654acef11c32670b534670c3a06e483fce313d72e3e0a15baa8
drwx------ 3 root root 4096 Jun 20 16:11 55f1e14c361b90570df46371b20ce6d480c434981cbda5fd68c6ff61aa0a5358
drwx------ 3 root root 4096 Jun 20 16:11 824c8a961a4f5e8fe4f4243dab57c5be798e7fd195f6d88ab06aea92ba931654
drwx------ 3 root root 4096 Jun 20 16:11 ad0fe55125ebf599da124da175174a4b8c1878afe6907bf7c78570341f308461
drwx------ 3 root root 4096 Jun 20 16:11 edab9b5e5bf73f2997524eebeac1de4cf9c8b904fa8ad3ec43b3504196aa3801
</pre>
<ul>
<li>The organization of files allows for efficient use of disk space.</li>
<li>There are <b>files unique to every layer</b> and <b>hard links to files</b> shared with lower layers
</li>
</ul>
</ul>
</ul>
</div>
<div class="slide">
<hgroup>
<h1>Dockerfile</h1>
</hgroup>
<ul class="xx-small" style="zoom: 0.90">
<li>Dockerfile is a script that creates a new image</li>
<pre class="brush: plain; gutter: 'false'">
# This is a comment
FROM oraclelinux:7
MAINTAINER Tomas Vitvar <tomas@vitvar.com>
RUN yum install -q -y httpd
EXPOSE 80
CMD httpd -X
</pre>
<li>A line in the Dockerfile will create an intermediary layer</li>
<pre class="brush: plain; gutter: 'false'">
$ docker build -t tomvit/httpd:v1 .
Sending build context to Docker daemon 2.048 kB
Step 1 : FROM oraclelinux:7
---> 4c357c6e421e
Step 2 : MAINTAINER Tomas Vitvar <tomas@vitvar.com>
---> Running in 35feebb2ffab
---> 95b35d5d793e
Removing intermediate container 35feebb2ffab
Step 3 : RUN yum install -q -y httpd
---> Running in 3b9aee3c3ef1
---> 888c49141af9
Removing intermediate container 3b9aee3c3ef1
Step 4 : EXPOSE 80
---> Running in 03e1ef9bf875
---> c28545e3580c
Removing intermediate container 03e1ef9bf875
Step 5 : CMD httpd -X
---> Running in 3c1c0273a1ef
</pre>
<ul class="no-bullet">
<li>If processing fails at some step, all preceeding steps will be loaded from the cache on the next run.
</li>
</ul>
</ul>
</div>
<div class="slide">
<hgroup>
<h1>Docker Container State Diagram</h1>
</hgroup>
<div class="h-drawing" style="width: 780px" id="1JGxpdEWDh9lBbLg3JFVr53LBg1BM520LYXyYkdi-2ws"></div>
</div>
</section>