The Command Line?
-
It’s difficult
-
Only nerds do it!
-
It’s obsolete, better use my super GUI
-
I don’t have that on my computer
The Command Line?
-
Using the command line interface is a must to control your computer
-
It’s a rich environment with time-proof tools
-
It’s always been available on all operating systems!
-
All activities in the Computer Science or similar courses make use of the command line
-
It is a pain to learn … if you don’t want to!
-
Even if you want to, the learning curve may be steep…
-
Objectives
-
Understanding the filesystem
-
Discovering and practising the most common Linux commands
-
Discovering and practising the input and output streams as well as pipelines
-
Using variables and creating scripts
-
Discovering Regular Expressions
Big Data ?
Process large volumes of data quickly and automatically - that’s the promise of Big Data.
Data often comes from the real world (production lines, sensors in vehicles, sensors in buildings, etc.) and is generally raw.
This data needs to be cleaned before it can be analyzed, understand the system’s behavior, etc.
This tedious cleaning work can be lightened by using the right tools.
Content
-
Very light presentations
-
Few minutes introducing the topic
-
-
Lots of hands-on work where you experiment, you learn
-
Work on raw data
-
-
Create simple, but well-understood commands, responding to a precise but generalizable problem
-
Favor one liners, but do not shy away from more complex scripts
-
From the most simple commands to full-featured scripts
Working Environment
We favor the use of Linux, in all its forms.
-
A Linux distribution (Debian, Ubuntu, OpenSuse, …)
-
WSL (Windows Subsystem for Linux)
-
MacOS + Terminal
Working Environment
-
Proven tools
-
Use of the command line
-
Concepts found everywhere
-
More freedom than with
graphicaltools
Windows Subsystem for Linux
-
Introduced in 2016 (Windows 10, Windows server 2016)
-
Available on more recent Microsoft OSes
-
A virtual machine (Hyper-V) with a Linux kernel
-
Allows Linux distributions to be installed in containers
-
Integration with Windows
MacOS
-
Natively a
Unixoperating system (BSD) -
Terminal gives access to all classic commands
-
Some differences in behavior between BSD and Linux commands
References
-
"Unix Text Processing", Dale Dougherty and Tim O’Reilly, Hayden Books, 1987
https://www.oreilly.com/openbook/utp/ -
Christophe Blaess cheat sheets[FR]
https://www.blaess.fr/christophe/developpements/aides-memoires/-
Unix commands[FR]
https://www.blaess.fr/christophe/memo_commandes_unix.html -
Shell programming[FR]
https://www.blaess.fr/christophe/memo_programmation_shell.html
-
-
Rich’s sh (POSIX shell) tricks
https://www.etalabs.net/sh_tricks.html -
Bash Reference Manual
https://www.gnu.org/software/bash/manual/bashref.html
-
Advanced Bash-Scripting Guide
http://tldp.org/LDP/abs/html/ -
"Mastering Regular Expressions, 3rd Edition — Understand Your Data and Be More Productive", Jeffrey Friedl
https://www.oreilly.com/library/view/mastering-regular-expressions/0596528124/ -
"GAWK: Effective AWK Programming", Edition 4.1
http://www.gnu.org/software/gawk/manual -
Manual pages‡ :
bash(1),grep(1),regex(7),gawk(1)
‡ : read thoses pages on your own operating system, not on the Internet!