-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathtutorial.html
More file actions
418 lines (399 loc) · 19.5 KB
/
Copy pathtutorial.html
File metadata and controls
418 lines (399 loc) · 19.5 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="color-scheme" content="light dark">
<title>Getting Started — Burzynski Lab</title>
<meta name="description" content="Getting started guide for the Burzynski group: OSCER, Athena, and storage.">
<link rel="canonical" href="https://jacksonburzynski.web.cern.ch/tutorial.html">
<link rel="icon" href="/favicon.svg" type="image/svg+xml">
<link rel="icon" href="/favicon-32.png" sizes="32x32" type="image/png">
<link rel="apple-touch-icon" href="/apple-touch-icon.png">
<link rel="preconnect" href="https://1.www.s81c.com" crossorigin>
<link rel="stylesheet" href="https://1.www.s81c.com/common/carbon/plex/sans.css">
<link rel="stylesheet" href="https://1.www.s81c.com/common/carbon/plex/mono.css">
<link rel="stylesheet" href="https://1.www.s81c.com/common/carbon/web-components/tag/v2/latest/themes.css">
<link rel="stylesheet" href="/css/site.css?v=14">
<script type="module" src="https://1.www.s81c.com/common/carbon/web-components/tag/v2/latest/tabs.min.js"></script>
</head>
<body class="cds-theme-zone-white">
<script>
/* Theme: follow the browser/system preference, with a manual override
stored in localStorage (set by the header toggle) */
(function () {
var m = window.matchMedia('(prefers-color-scheme: dark)');
function current() {
var stored = null;
try { stored = localStorage.getItem('theme'); } catch (e) {}
return stored || (m.matches ? 'dark' : 'light');
}
function apply() {
document.body.className = current() === 'dark' ? 'cds-theme-zone-g100' : 'cds-theme-zone-white';
}
apply();
m.addEventListener('change', apply);
document.addEventListener('click', function (e) {
if (e.target.closest && e.target.closest('.theme-toggle')) {
var next = current() === 'dark' ? 'light' : 'dark';
try { localStorage.setItem('theme', next); } catch (e2) {}
apply();
}
});
/* Narrow screens: keep the current page's nav item in view */
document.addEventListener('DOMContentLoaded', function () {
var nav = document.querySelector('.site-header__nav');
var cur = nav && nav.querySelector('[aria-current="page"]');
if (!nav || !cur) return;
var offset = cur.offsetLeft - (nav.clientWidth - cur.offsetWidth) / 2;
if (offset > 0) nav.scrollLeft = offset;
});
})();
</script>
<a class="skip-link" href="#main">Skip to main content</a>
<header class="site-header">
<div class="site-header__inner container">
<a class="site-header__name" href="/">Prof. Jackson Burzynski</a>
<nav class="site-header__nav" aria-label="Site">
<a href="/">Home</a>
<a href="/research.html">Research Overview</a>
<a href="/tutorial.html" aria-current="page">Tutorial</a>
<a href="/meetings.html">Meetings</a>
<a href="/news.html">News</a>
<a href="/links.html">Links</a>
<a href="/teaching.html">Teaching</a>
</nav>
<button class="theme-toggle" type="button" aria-label="Toggle light and dark mode">
<svg class="icon-moon" viewBox="0 0 32 32" aria-hidden="true"><path d="M13.502 5.414a15.075 15.075 0 0 0 11.594 18.194 11.113 11.113 0 0 1-7.975 3.39c-.138 0-.278.005-.418 0a11.094 11.094 0 0 1-3.2-21.584M14.98 3a1.002 1.002 0 0 0-.175.016 13.096 13.096 0 0 0 1.825 25.981c.164.006.328 0 .49 0a13.072 13.072 0 0 0 10.703-5.555 1.01 1.01 0 0 0-.783-1.565A13.08 13.08 0 0 1 15.89 4.38 1.015 1.015 0 0 0 14.98 3Z"/></svg>
<svg class="icon-sun" viewBox="0 0 32 32" aria-hidden="true"><path d="M16 12.005a4 4 0 1 1-4 4 4.005 4.005 0 0 1 4-4m0-2a6 6 0 1 0 6 6 6 6 0 0 0-6-6ZM5.394 6.813 6.81 5.399l3.505 3.506L8.9 10.319zM2 15.005h5v2H2zm3.394 10.193L8.9 21.692l1.414 1.414-3.505 3.506zM15 25.005h2v5h-2zm6.687-1.9 1.414-1.414 3.506 3.506-1.414 1.414zm3.313-8.1h5v2h-5zm-3.313-6.101 3.506-3.506 1.414 1.414-3.506 3.506zM15 2.005h2v5h-2z"/></svg>
</button>
</div>
</header>
<main id="main">
<div class="container">
<div class="page-head">
<h1 class="page-title">Getting Started</h1>
<p class="page-subtitle">
A guide for new members of the group. Under construction —
more detail coming soon.
</p>
</div>
<noscript><style>.doc--tabs section[hidden]{display:block}cds-tabs{display:none}</style></noscript>
<cds-tabs value="basics" class="doc-tabs">
<cds-tab id="tab-basics" value="basics" target="panel-basics">Basics</cds-tab>
<cds-tab id="tab-oscer" value="oscer" target="panel-oscer">OSCER</cds-tab>
<cds-tab id="tab-analysis" value="analysis" target="panel-analysis">Python</cds-tab>
<cds-tab id="tab-athena" value="athena" target="panel-athena">ATLAS</cds-tab>
</cds-tabs>
<div class="doc doc--tabs">
<section id="panel-basics" role="tabpanel" aria-labelledby="tab-basics">
<h2>Basics</h2>
<p>
Before touching a cluster, set up a comfortable local environment.
You will spend most of your time in a terminal and an editor, so it
is worth getting these right on day one.
</p>
<h3>Terminal</h3>
<p>
On macOS, install <a href="https://iterm2.com/" rel="noopener">iTerm2</a>
— a far more capable replacement for the built-in Terminal
(split panes, search, better copy/paste). If you use
<a href="https://brew.sh/" rel="noopener">Homebrew</a> (recommended
for managing software on macOS):
</p>
<pre><code>brew install --cask iterm2</code></pre>
<p>
On Windows, use
<a href="https://learn.microsoft.com/en-us/windows/wsl/install" rel="noopener">WSL</a>
with Windows Terminal so you have a real Linux environment.
</p>
<h3>Editor</h3>
<p>
Install <a href="https://code.visualstudio.com/" rel="noopener">VS Code</a>
and add the <b>Remote — SSH</b> extension. This lets you edit
files living on OSCER or lxplus directly from your laptop, with
full syntax highlighting and search — much nicer than editing
over a terminal editor for large projects. (That said, learn enough
<code>vim</code> to make quick edits on any machine.)
</p>
<h3>Git</h3>
<p>
All of our code lives in git repositories on
<a href="https://github.com/" rel="noopener">GitHub</a> and
<a href="https://gitlab.cern.ch/" rel="noopener">CERN GitLab</a>.
Configure your identity once per machine:
</p>
<pre><code>git config --global user.name "Your Name"
git config --global user.email "you@ou.edu"</code></pre>
<p>
Generate an SSH key (<code>ssh-keygen -t ed25519</code>) and add
the public key (<code>~/.ssh/id_ed25519.pub</code>) to both GitHub
and CERN GitLab so you can push without typing passwords.
</p>
<h3>Learning resources</h3>
<p>
The <a href="https://hepsoftwarefoundation.org/training/curriculum.html" rel="noopener">HSF
Training Center</a> has excellent self-paced material on the shell,
git, Python, and the rest of the HEP software toolkit. If you are
new to any of these, start there.
</p>
</section>
<div id="panel-oscer" role="tabpanel" aria-labelledby="tab-oscer" hidden>
<section>
<h2>OSCER account</h2>
<p>
Our group computes on OSCER, the OU Supercomputing Center for
Education & Research. Steps to get an account:
</p>
<ol>
<li>Complete the <a href="https://ousurvey.qualtrics.com/jfe/form/SV_exMG0fbmXp1YWrz" rel="noopener">New Account Request Form</a>.</li>
<li>If approved, you will receive an email when your account is created.</li>
<li>Log in with your initial password and <a href="https://www.ou.edu/oscer/getting-started/change-password.html/#change_password" rel="noopener">change it</a> IMMEDIATELY if prompted (or manually if not prompted).</li>
</ol>
<p>
The <a href="https://www.ou.edu/oscer/getting-started" rel="noopener">OSCER
documentation</a> is the reference for anything not covered on
this page. For account problems, quota issues, or anything else
cluster-related, contact
<a href="mailto:support@oscer.ou.edu">support@oscer.ou.edu</a>
(see the <a href="https://www.ou.edu/oscer/support" rel="noopener">OSCER
support page</a>).
</p>
<h3>Logging in</h3>
<p>Once your account is active, connect with:</p>
<pre><code>ssh -XY [username]@sooner.oscer.ou.edu</code></pre>
<p>
To save typing, add an entry to <code>~/.ssh/config</code> on your
laptop:
</p>
<pre><code>Host sooner
HostName sooner.oscer.ou.edu
User [username]
ForwardX11 yes
ForwardX11Trusted yes</code></pre>
<p>after which <code>ssh sooner</code> is all you need.</p>
</section>
<section>
<h2>Requesting a node on Sooner</h2>
<p>
The group has a dedicated Slurm partition, currently named
<code>ouheptmp</code> (this will change to <code>ouhep</code> in
the future).
<b>Never run real work on the login nodes</b> — they are
shared by everyone and only meant for editing files and submitting
jobs. Always run your code on an interactive node. To request one:
</p>
<pre><code>srun -p sooner_test --container=el9hw --pty $SHELL</code></pre>
<p>
This drops you into a shell on a compute node inside an EL9
container, which is what ATLAS software expects.
</p>
<h3>GPU access</h3>
<p>
For GPU work (e.g. machine learning training), request a GPU node
with:
</p>
<pre><code>srun -p ouheptmp --container=el9hw --gres=gpu:1 --ntasks=1 --cpus-per-task=16 --mem=40G --pty $SHELL</code></pre>
<p>
Check what you were allocated with <code>nvidia-smi</code> once
the shell opens.
</p>
</section>
<section>
<h2>Submitting batch jobs</h2>
<p>
For anything longer than a quick test, submit a batch job instead
of holding an interactive shell. Write a submission script like:
</p>
<pre><code>#!/bin/bash
#SBATCH --partition=ouheptmp
#SBATCH --container=el9hw
#SBATCH --job-name=myjob
#SBATCH --output=logs/%x-%j.out
#SBATCH --time=12:00:00
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=4
#SBATCH --mem=8G
setupATLAS
asetup Athena,main,latest
# your commands here</code></pre>
<p>and submit it with:</p>
<pre><code>sbatch myjob.sh</code></pre>
<p>Useful commands for managing your jobs:</p>
<pre><code>squeue -u $USER # list your queued and running jobs
scancel [jobid] # kill a job
sacct -j [jobid] # accounting info for a finished job</code></pre>
<p>
<code>%x</code> and <code>%j</code> in the output path expand to
the job name and job ID. Make sure the <code>logs/</code>
directory exists before submitting. Request only the resources you
need — smaller jobs schedule faster.
</p>
</section>
<section>
<h2>Storage: OURDisk</h2>
<p>
The group's OURDisk partition lives at:
</p>
<pre><code>/ourdisk/hpc/ouhep/</code></pre>
<p>
We have a shared quota of 93 TB. Please be mindful not to
fill it up — clean up datasets you no longer need, and check
with the group before copying in anything large.
</p>
</section>
</div>
<section id="panel-athena" role="tabpanel" aria-labelledby="tab-athena" hidden>
<h2>Setting up Athena</h2>
<p>
ATLAS software is distributed via CVMFS, which is mounted on the
OSCER nodes. First, add this to your <code>~/.bashrc</code>:
</p>
<pre><code>export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase
alias setupATLAS='source $ATLAS_LOCAL_ROOT_BASE/user/atlasLocalSetup.sh'</code></pre>
<p>
Then, on an interactive node (see above), initialize the ATLAS
environment and set up an Athena release:
</p>
<pre><code>setupATLAS
asetup Athena,main,latest</code></pre>
<p>
Use <code>Athena,main,latest</code> for development against the
main branch. For analysis, you will usually pin a specific release
(e.g. <code>asetup Athena,25.0.25</code>) so results are
reproducible.
</p>
<h3>Working with the Athena repository</h3>
<p>
Athena development uses a sparse checkout: you check out only the
packages you are changing, and build them against the release.
First <a href="https://gitlab.cern.ch/atlas/athena/-/forks/new" rel="noopener">fork
the athena repository</a> on CERN GitLab (once), then:
</p>
<pre><code>lsetup git
git atlas init-workdir ssh://git@gitlab.cern.ch:7999/[username]/athena.git
cd athena
git atlas addpkg MyPackage
git checkout -b main-my-topic upstream/main --no-track</code></pre>
<p>
where <code>MyPackage</code> is the package you want to modify
(add as many as you need with further <code>git atlas addpkg</code>
calls). Build the checked-out packages in a separate build
directory:
</p>
<pre><code>mkdir ../build && cd ../build
cmake ../athena/Projects/WorkDir
make -j8
source x86_64-*/setup.sh</code></pre>
<p>
After sourcing the setup script, your locally built packages
shadow the ones in the release. When your changes are ready,
push your branch to your fork and open a merge request to
<code>atlas/athena</code>.
</p>
<p>
For a full walkthrough, see the
<a href="https://atlas-software.docs.cern.ch/analysis/analysis_tutorial/" rel="noopener">ATLAS software tutorial</a>
and the <a href="https://atlas-software.docs.cern.ch/athena/git/" rel="noopener">ATLAS git workflow tutorial</a>.
</p>
</section>
<div id="panel-analysis" role="tabpanel" aria-labelledby="tab-analysis" hidden>
<section>
<h2>Python environments with mamba</h2>
<p>
Manage Python with
<a href="https://mamba.readthedocs.io/" rel="noopener">mamba</a>,
a fast drop-in replacement for conda. On OSCER, mamba is already
available as a module — do not install your own. Load it
with:
</p>
<pre><code>module load Mamba</code></pre>
<p>
(The first time, run <code>mamba init bash</code> and log out and
back in so environment activation works.) Then create one
environment per project — never install packages into the
base environment:
</p>
<pre><code>mamba create -n myanalysis python=3.12
mamba activate myanalysis</code></pre>
<p>
Environments keep each project's dependencies isolated and
reproducible. Note that environments can get large — if your
home area fills up, contact
<a href="mailto:support@oscer.ou.edu">OSCER support</a>.
</p>
</section>
<section>
<h2>Machine learning on OSCER</h2>
<p>
For machine learning work, create a dedicated environment and
install <a href="https://pytorch.org/" rel="noopener">PyTorch</a>
and <a href="https://lightning.ai/docs/pytorch/stable/" rel="noopener">Lightning</a>
into it (the PyTorch wheels bundle CUDA, so the same install works
on CPU and GPU nodes):
</p>
<pre><code>mamba create -n ml python=3.12
mamba activate ml
pip install torch lightning</code></pre>
<p>
Then request a GPU node (see the OSCER tab) and check that the GPU
is visible:
</p>
<pre><code>python -c "import torch; print(torch.cuda.is_available())"</code></pre>
<p>
Do the installation itself from an interactive node rather than a
login node — the downloads are large. Add whatever else your
project needs (<code>numpy</code>, <code>matplotlib</code>,
<code>jupyterlab</code>, …) to the same environment.
</p>
</section>
<section>
<h2>HEP data analysis: the Scikit-HEP stack</h2>
<p>
For ntuple-level analysis we recommend the
<a href="https://scikit-hep.org/" rel="noopener">Scikit-HEP</a>
ecosystem: modern, columnar Python analysis that plays well with
the broader scientific Python world (NumPy, matplotlib, machine
learning frameworks) — no compiled ROOT macros required.
The core pieces:
</p>
<ul>
<li><a href="https://uproot.readthedocs.io/" rel="noopener"><code>uproot</code></a> — reads and writes ROOT files directly into arrays, with no ROOT installation needed</li>
<li><a href="https://awkward-array.org/" rel="noopener"><code>awkward</code></a> — NumPy-style operations on jagged arrays (variable-length collections like jets per event)</li>
<li><a href="https://hist.readthedocs.io/" rel="noopener"><code>hist</code></a> — histograms with a clean API for filling, slicing, and projecting</li>
<li><a href="https://mplhep.readthedocs.io/" rel="noopener"><code>mplhep</code></a> — matplotlib styles and helpers for publication-quality HEP plots (including the ATLAS style)</li>
<li><a href="https://vector.readthedocs.io/" rel="noopener"><code>vector</code></a> — Lorentz vectors and kinematics on arrays</li>
</ul>
<p>Install them into your environment from conda-forge:</p>
<pre><code>mamba install uproot awkward hist mplhep vector jupyterlab</code></pre>
<p>A minimal analysis looks like:</p>
<pre><code>import uproot
import awkward as ak
import hist
import mplhep as hep
import matplotlib.pyplot as plt
tree = uproot.open("ntuple.root")["tree"]
jet_pt = tree["jet_pt"].array() / 1000.0 # MeV to GeV
h = hist.Hist.new.Reg(50, 0, 500, label="Jet $p_{T}$ [GeV]").Double()
h.fill(ak.flatten(jet_pt))
hep.style.use(hep.style.ATLAS)
h.plot()
plt.savefig("jet_pt.pdf")</code></pre>
<p>
The <a href="https://hsf-training.github.io/hsf-training-scikit-hep-webpage/" rel="noopener">HSF
Scikit-HEP training module</a> is a good hands-on introduction to
the whole stack.
</p>
</section>
</div>
</div>
</div>
</main>
<footer class="site-footer">
<div class="site-footer__inner container">
<span>© 2026 Jackson Burzynski</span>
<span><a href="mailto:burzynski@ou.edu">burzynski@ou.edu</a></span>
</div>
</footer>
</body>
</html>