Intro to Dev Ops

class: center, middle

# Intro to Dev Ops

---

## Please Take Notes
* We are going to cover a lot of topics, techs, and commands.
* You will want to research them later.

---

### A Brief History of Ops

In the past, Hardware was:
* Very expensive
* Ran onsite in data centers
* Required months of lead to planning and acquisition
* Required significant expertise to configure, monitor, and maintain
* Scaled vertically (bigger machines)

####This naturally led to an `Ops` team.
* System administrators
* Upgraded systems
* Planned for capacity
* Kept systems running

---

### A Brief History of Ops

However, starting in the early 2000s computing began to change:
* Computing began to scale horizontally (more machines)
* New Way:  If you needed more memory, CPU, etc. you add another machine.
* (Old Way:  If you needed more memory, CPU, etc. you threw away your current machine and bought a bigger one)

<img src="googles-first-server.jpg"></img>
<div style = "text-align:center">Google's First Production Servers</div>

---

### A Brief History of Ops

With horizontally scalable computing power, it then became possible to:
* Build systems that can accommodate massive scale
* Share / Rent your computational capacity to others

### In 2006, Amazon launched the EC2 and S3 product.
* Amazon had developed significant computational power for serving Amazon.com
* EC2 allowed you to rent a virtualized machine that ran on their infrastructure

---

### A Brief History of Ops

Amazon EC2 (and similiar products) were the start of Cloud Computing.

Cloud Computing was a game changer because:
* Virtualized computers could be rented from systems that scale horizontally.
* Virtualized computers are on demand.
* Virtualized computers are cheap because they use commodity hardware.
* Virtualized computers are hands off because they run in a remote data center.
* Virtualized computers are (relatively) easier to configure as a single machine.
* Virtualized computers have APIs which means that you can write code to spin up a machine.

---

## DevOps Puts the Developer in the Operations

Cloud Virtualized Computing made operations accessible to the developer.

DevOps is a culture of developers working close to operations.

This is good for the developer because:
* You no longer have to wait on operations to provide hardware.
* You can more easily identify bugs because you can exactly replicate a production environment.
* You can design your systems for scalability from the beginning.

---

## DevOps in the Workplace

Developers are encouraged to own the lifecycle of their application.

* The app that you write locally needs to be deployable on a server.
* The app that you write locally also needs to go through a series of checks:
  * All tests need to pass.
  * All linters need to pass.
  * All security checks need to pass.
* The app that you write locally needs to be able to scale under load.
* When there is a problem with your app, you will need to be able to discover the issue.

---

## This Week

You will be learning how to manage your app's pipeline.

1. Basic Ops - How to deploy an app on a remote server and troubleshoot issues.
2. Ops in a Cloud Environment - How to deploy apps that are secure and scale in the cloud.
3. Best Practices of Cloud Ops - How to make your deployed app as simple and reliable as possible.
4. Build Systems - You will learn more about Gradle and how it builds your project.

---

## Basic Ops

In order to do basic ops, it is important to understand the following topics:
1. Building code to run in "production"
2. Accessing remote machines
3. Unix systems and permissions
4. Networking and Ports
5. App Troubleshooting

---

## Building Code to Run in the Cloud

Right now you are used to running your code from IntelliJ IDEA.

This works great for development, but there are important changes required for code to be run in "production":

1. Code has to be packaged to run on a server (Java is JAR or WAR file).
2. Code is optimized or obfuscated.
3. System credentials have to be changed.

---

## What Do You Mean, JAR and WAR?

Java Archive File and Web Archive file (respectively)

* Both are ZIP files whose contents are arranged in a particular order.
* (Unzip a jar file some time and you can see what is inside.)
* Contains all of the compiled code required to run a program.

**Difference**

JAR Files are typically used as libraries so that you can include in other projects.

WAR Files are deployed on a web server like Tomcat, Weblogic, etc.

---

## Executable Jar Files

**There are also executable jar files**

```
java -jar myapp.jar

```

**Hint**

Spring Boot comes with the ability to create an executable Jar file with the web server baked in.

This means that you can put the Jar file on any machine that has Java and you will have a web app!
---

## SSH: Accessing Remote Machines

SSH allows you to gain access to a remote computer's command line.

```
ssh mike@192.38.233.11
```

(which is saying)
```
ssh {username}@{ip address} #connect to ip address with username
```

---

## SSH Stands for **SECURE** Shell

SSH uses public key encryption to create a secure connection between your computer and the remote computer.

* Each computer participating in SSH stores a public key and a private key in the `~/.ssh` directory.

* Each computer shares the public key.

* Each computer keeps the private key strictly secret.

* Using a [Diffie-helman key exchange](https://en.wikipedia.org/wiki/Diffie%E2%80%93Hellman_key_exchange), a shared secret is generated which can be used as a key to encrypt traffic between the two clients.

---

## SSH Authentication Types

*No matter what, you need to have public private keys in your `~/.ssh` directory to use SSH.*

**Password Authentication** - the username and password for a user on that machine can be used in SSH.  
* Highly insecure because it is vulnerable to brute force attacks

**Key-based Authentication** - only certain public keys are allowed to access a specific machine.
* Secure. The combination of the public key and the private key provides access.
* Allowed keys are stored in the `~/.ssh/known_hosts` file.
* Example - GitHub allows you to provide a public key that you can use to autheticate your code pushes.

---

## Unix Systems

Unix operating systems are designed to support many users accessing one machine.

* Anyone that wants to access a machine has a username and password.
* Each user "owns" a specific portion of the file system called the Home Directory.
```
/home/{username}
```

---

## Unix Users

* Each user is part of a group.
* Every file specifies which users or which groups can access the file.
```
$ ls -la
drwxrwxr-x+ 56 root  admin  1904 Jan 30 11:58 Applications
drwxr-xr-x+ 62 root  wheel  2108 Nov  7 13:28 Library
...
```
---

## Unix: File Access

Take a look at how file access is configured.

```
$ ls -la
drwxrwxr-x+ 56 root  admin  1904 Jan 30 11:58 Applications
drwxr-xr-x+ 62 root  wheel  2108 Nov  7 13:28 Library
...
```

**r** - provides read access
**w** - provides write access
**x** - provides execute access

```
drwx------+ ... # provides rwx to the owner 
d---rwx---+ ... # provides rwx to the group 
d------rwx+ ... # provides rwx to the others 
```

---

## Unix: More Permissions

Who are the Owner, the Group, and Others???

```
$ ls -la
drwxrwxr-x+ 56 root  admin  1904 Jan 30 11:58 Applications
drwxr-xr-x+ 62 root  wheel  2108 Nov  7 13:28 Library
...
```
**root** is the Owner

**admin** is the Group

*__Others__ is everyone else!*

---

## Unix: Permission Examples

```
$ ls /home/skywalker
drwxr-xr-x   11 skywalker staff    374 Sep 30 15:43 Code
drwx------+  68 skywalker staff   2312 Jan 31 20:51 Desktop
drwxrwx---+  23 skywalker staff    782 Jan 30 11:58 Documents
```

`~/home/skywalker/Desktop`
* Only `skywalker` can read, write, and execute files

`~/home/skywalker/Documents`
* `skywalker` and any user in the group `staff` can read, write, and execute fles

*Anyone on the computer can execute programs in `~/home/skywalker/Code`.*

---

# Unix is designed to provide users the **minimal amount** of access to the system

---

## Unix: Permissions for All!

All users on a Unix system have a username and a home directory.

All **processes** on a Unix system also have a user.

---

## Unix Processes

*Everything happening on your Unix machine is running as a process.*

```
ps -o pid,ppid,tty,uid,args 
  PID  PPID TTY        UID ARGS
 1234  1233 ttys000    501 -bash
 8119  1234 ttys000    501 python -m SimpleHTTPServer
 5294   287 ttys002    501 /bin/bash --rcfile /Applications/IntelliJ IDEA CE.app/Contents/p
```
**pid** - the unique identifier for the process

**ppid** - the id of the parent process

**tty** - the terminal session that triggered the process

**uid** - the user id that owns the process

**args** - what process is running

---

## Unix Processes

### Normal Processes

* Every process has a parent.
* A process either comes from a **fork** of an existing process
* Or **exec** replaces the existing process 
* *Owned by the user that initiated the process*

### Daemon Processes

Daemon processes are special processes that are run in the background.

(e.g. Databases, webservers, etc.)

*Daemon processes need to be assigned their own user*.

---

## Working with Daemon Processes

Daemon processes can be started in several ways:

* Using an ampersand at the end of a command
```
$ kibana &
```

* Start through `init` or `systemd`
```
$ sudo service elasticsearch start
```

---

## **systemd** for Server Startup

**Systemd** is a Linux tool that allows you to manage **units** (i.e. services, sockets, devices, etc.).

* Systemd provides an easy way to interface with services.
* Systemd allows more finegrained control over how a process runs.
```
$ sudo systemctl {processname} start  # start server in background 
$ sudo systemctl {processname} restart 
$ sudo systemctl {processname} stop
$ sudo systemctl {processname} enable  # auto start on boot
$ sudo systemctl {processname} disable # auto end on boot
```

---

## **systemd** Configuration

Systemd is configured by placing a script in the `/etc/systemd/system` directory.

*Note: User lets you run the process as a specific user on your system*
`/etc/systemd/system/{your-app}.service`
```
[Unit]
Description=A Spring Boot application
After=syslog.target
 
[Service]
User=app_user
ExecStart=/path/to/your-app.jar SuccessExitStatus=143 
 
[Install] 
WantedBy=multi-user.target
```

---

## **jounalctl** Logs

Journalctl is a command that makes it easy to access the logs of your service.

*Typically log files are accessed in the `/var/log` directory*.

```
$ journalctl      # view all logs (oldest first)
$ journalctl -b   # view logs from current boot
$ journalctl -u nginx.service  # all logs from nginx 
$ journalctl -b -u nginx.service  # all logs from nginx boot 
```
---

## **systemd** Configuration

*Remember, your app needs to run as its own user for security reasons*.

1. Create a user and set the password

```
$ sudo useradd app_user
$ sudo passwd some_solid_password
```

2. Set permissions and ownership of your app:

```
$ sudo chown app_user:app_user_group your-app.jar
$ sudo chmod 500 your-app.jar

```
---

## **systemd** Configuration

Then start it up and see it run. Well you won't be able to see it run, because it will be running in the background.

```
$ sudo service your-app start

```

To see if it worked you will have to look at the logs via

```
jounalctl -f -u your-app
```
---

## **Not** Working with Daemon Processes

If need to stop a daemon process that you already started:

1. Find the process that you want to go away.
```
$ ps aux | grep name_of_process_to_go_away
```
2. Get the PID of the process.
```
$ ps -o pid,ppid,tty,uid,args 
  PID  PPID TTY        UID ARGS
 8119  1234 ttys000    501 python -m SimpleHTTPServer # PID 8119
```

---

## **Not** Working with Daemon Processes

(3) Send the signal to stop it.
```
$ kill -9 8119 
```

(4) Check to make sure it is gone.
```
$ ps aux | grep name_of_process_to_go_away
```

---

##Important Paths from Root:

* `/home/` - home directory of all users 
* `/opt/` - third party packages that don't conform to Linux hierarchy
* `/var/` - variable data, **logs**, often changed files
* `/bin/` - common binaries
* `/sbin/` - binaries used by the system administrators<
* `/boot/` - static files on the bootloader (i.e. Linux kernel)
* `/dev/` - devices such as CD drive
* `/lib/` - essential libraries and kernel modules
* `/mnt/` - temporary mount (i.e. for USB)
* `/tmp/` - temporary files

---