Ansible - Ansible Vault - keep your password secret

 ================Ansible Vault==================
1. Create your yaml file
We are going to create a file keepitsecret.yaml and we will keet it secret using vault

[root@master vault]# cat myvault.yaml
- hosts: 127.0.0.1
  vars_files:
    - keepitsecret.yaml
  tasks:
  - name: Sending email using Gmail's smtp services
    mail:
      host: smtp.gmail.com
      port: 587
      username: "{{ u }}"
      password: "{{ p }}"
      to: sam@gmail.com
      subject: Testing email from gmail using ansible
      body: system {{ ansible_host }} has been successfully tested.

2. Create a vault where you will store your username/pw
# av -h
# av create -h
check the syntax
[root@master vault]# ansible-vault create keepitsecret.yaml
New Vault password:
Confirm New Vault password:
u: "sam@gmail.com"
p: "MyPasswordSecret"

3. View the content of the file. You can't read what your stored. Its encripted.
[root@master vault]# cat keepitsecret.yaml
$ANSIBLE_VAULT;1.1;AES256
32346435633239646636626465663162613262623434333664393437316461366565316364396632
6365373834616464333437373134653435386335653165660a326331363163353932373161386362
61316464353339383834666662353230393036313538646563303632393134363165353431336130
3037393363643463650a643762353433663662306630376231363836376464656330346235663964
31656463373832353739303239353032613838333231613464343336656239656535333561663064
3036336665303135313061666234313831626630343066613130
[root@master vault]#

4. Run your playbook
# ap myvault.yaml
I got email alert
Sign-in attempt was blocked
sam@gmail.com
Someone just used your password to try to sign in to your account from a non-Google app. Google blocked them, but you should check what happened. Review your account activity to make sure no one else has access.

Less secure app blocked
Google blocked the app you were trying to use because it doesn't meet our security standards.
Some apps and devices use less secure sign-in technology, which makes your account more vulnerable. You can turn off access for these apps, which we recommend, or turn on access if you want to use them despite the risks. Google will automatically turn this setting OFF if it's not being used.
Learn more
google for less secure app access and 
Enabling less secure apps to access Gmail

you should be send email this time.

Ansible: exception handling ... error handling ..

Ansible: exception handling ...

1. Lets create a simple playbook and run

[root@master dec15]# cat error.yaml

- hosts: w1

  tasks:

  - package:

      name: "httpd"

      state: present

  - debug:

      msg: "This is a test run .."

[root@master dec15]# ansible-playbook error.yaml

2. Lets make a mistake on one of the variable.

say pkg name is nane

# cat error.yaml

- hosts: w1

  tasks:

  - package:

        nane: "httpd"

        state: present

- debug:

    msg: "This is just a test run ..."

[root@master dec15]# alias 'ap=ansible-playbook'

[root@master dec15]# ap error.yaml

we saw fatal error. we know its because the keyword we wrote, ansible does not that that keyword defined.

in fact it didn't recognize the parameter e=we supply.

3. What we can do is tell ansible to ignore if you find error. do not just throw error, continue.

# cat error.yaml

- hosts: w1

  tasks:

  - package:

        nane: "httpd"

        state: present

    ignore_errors: yes # ignore this error and go to next task

- debug:

    msg: "This is just a test run ..."

[root@master dec15]# ap error.yaml

You see, error is ignored this time.

4. Now, lets try something else, lets download a file from internet..

[root@master dec15]# ansible-doc -l uri

# cat errors.yaml

- hosts: w1

  tasks:

    - package:

        name: "httpd"

        state: present

debug: 

   msg: "This is just testing msg"

Before running the playbook to download a file from internet, lets look into some docs

or google for ansible uri or get-uri

look for example 

Image source: https://hips.hearstapps.com/hmg-prod.s3.amazonaws.com/images/quotes-about-change-1580499303.jpg

[root@master dec15]# cat error.yaml

- hosts: w1

  tasks:

  - package:

      nane: "httpd"

      state: present

    ignore_errors: yes

  #- get_uri:

  - uri:

      url: https://hips.hearstapps.com/hmg-prod.s3.amazonaws.com/images/quotes-about-change-1580499303.jpg

      dest: "/var/www/html/life_l.jpg"

  - debug:

      msg: "This is a test run .."

[root@master dec15]#

[root@master dec15]# ap error.yaml

[root@worker1 ~]# ls -l /var/www/html/life_l.jpg

-rw-r--r--. 1 root root 206868 Dec  8 05:10 /var/www/html/life_l.jpg

The result above shows that its successful.

5. Now, say we have a problem with internet connection, and if you run this playbook,, it will fail.

It will throw error, so how do you handle the error?

The best thing you can do is ignore the error. How? look at the yaml file below..

[root@master dec15]# netstat -rn

Kernel IP routing table

Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface

0.0.0.0         192.168.10.1    0.0.0.0         UG        0 0          0 enp0s3

for lab purpose, you can remove the 0.0.0.0

[root@master dec15]# cat error.yaml

- hosts: w1

  tasks:

  - package:

      nane: "httpd"

      state: present

    ignore_errors: yes

  #- get_uri:

  - uri:

      url: https://hips.hearstapps.com/hmg-prod.s3.amazonaws.com/images/quotes-about-change-1580499303.jpg

      dest: "/var/www/html/life_l.jpg"

    ignore_errors: yes

  - debug:

      msg: "This is a test run .."

[root@master dec15]# ap error.yaml

If you are disconnected from internet, it will fail with error unreachable network. 

so, you can use ignore_errors keyword to ignore the error.

it will continue to run it. But it might be an important piece of information, that you can't ignore it.

so,  you have to be very careful while dealing with ignore_errors.

6. Using block.

On block, your code in block and at the end, include rescue.

[root@master dec15]# ap error.yaml

/usr/lib/python3.6/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.2) or chardet (3.0.4) doesn't match a supported version!

  RequestsDependencyWarning)

PLAY [w1] *********************************************************************************************************************

TASK [Gathering Facts] ********************************************************************************************************

ok: [w1]

TASK [package] ****************************************************************************************************************

ok: [w1]

TASK [uri] ********************************************************************************************************************

changed: [w1]

TASK [service] ****************************************************************************************************************

changed: [w1]

TASK [debug] ******************************************************************************************************************

ok: [w1] => {

    "msg": "This is a test run .."

}

TASK [debug] ******************************************************************************************************************

ok: [w1] => {

    "msg": "This is a test run .."

}

PLAY RECAP ********************************************************************************************************************

w1                         : ok=6    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

[root@worker1 ~]# systemctl status httpd

● httpd.service - The Apache HTTP Server

   Loaded: loaded (/usr/lib/systemd/system/httpd.service; disabled; vendor preset: disabled)

   Active: active (running) since Tue 2020-12-08 06:01:59 EST; 42s ago

     Docs: man:httpd.service(8)

 Main PID: 59114 (httpd)

[root@worker1 html]# ls -ltr

total 212

-rw-r--r--. 1 root root      8 Dec  7 03:19 index.html

-rw-r--r--. 1 root root     12 Dec  8 05:51 webap.htm

-rw-r--r--. 1 root root 206868 Dec  8 05:53 life_l.jpg

[root@worker1 html]# cat webap.htm

This is cool

Ansible - EC2 instance creation using ansible

Ansible - EC2 instance creation using ansible..

1. Write your playbook

-> Collect all the manual steps to create an EC2 instance. Google for EC2 instance creation using ansible..

# cat aws-ec2.yaml

- hosts: localhost # 192.168.56.5 - use your own control node)

  tasks:

  - ec2_instance:

      region: us-east-1

      image_id: ami-04d29b6f966df1537

      instance_type: t2.micro

      #image: t2.micro

      vpc_subnet_id: subnet-e261d2ec

      security_group: sg-f5b18ad2

      key_name: kt-2020-k

      name: os_from_ansible

      state: present

      aws_access_key: AKIA6DEA42GA2PGZJ7G3

      aws_secret_key: 3IYF568qVJ8I#RZYnUV2OPG8/XDKVrhDfJRJPnbc

2. Run your playbook

[root@master wk-dec9]# ansible-playbook aws-ec2.yaml

PLAY [localhost] *****************************************************************************************

TASK [Gathering Facts] ***********************************************************************************

ok: [localhost]

TASK [ec2_instance] **************************************************************************************

fatal: [localhost]: FAILED! => {"changed": false, "msg": "Failed to import the required Python library (botocore or boto3) on master's Python /usr/bin/python3.6. Please read module documentation and install in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter"}

PLAY RECAP ***********************************************************************************************

localhost                  : ok=1    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

3. Review the error and Install boto3

[root@master wk-dec9]# pip3 install boto3

Successfully installed boto3-1.16.33 botocore-1.19.33 s3transfer-0.3.3 urllib3-1.26.2

4. Re-run your playbook

[root@master wk-dec9]# ansible-playbook aws-ec2.yaml

PLAY [localhost] *****************************************************************************************

TASK [Gathering Facts] ***********************************************************************************

ok: [localhost]

TASK [ec2_instance] **************************************************************************************

changed: [localhost]

PLAY RECAP ***********************************************************************************************

localhost                  : ok=2    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

[root@master wk-dec9]#

5. Playbook content ..

[root@master wk-dec9]# cat aws-ec2.yaml

- hosts: localhost # 192.168.56.4 - your own control node)

  tasks:

  - ec2_instance:

      region: us-east-1

      image_id: ami-04d29b6f966df1537

      instance_type: t2.micro

      #image: t2.micro

      vpc_subnet_id: subnet-e251d2ec

      security_group: sg-f7a18ad2

      key_name: kb-2020-key

      name: os_from_ansible

      state: present

      aws_access_key: AKIC6HXA42MR2PGZJ7G3

      aws_secret_key: 3IYF590qVJ8ISpZYnUV92PG8/XDKVrhHsJcMPnbc

Ansible - Setup and configure Load Balancer and Proxy using HAProxy-automatically using ansible

Configure LB and proxy using haproxy

Requirement:

1. Once server for load balancer

2. one or two servers for web servers

In my example I have three servers

Load Balancer: master  - 192.168.10.50

Web servers: worker1, worker2 - 192.168.10.51/52

1. On master server, Install haproxy - comes on RedHat DVD

# yum install haproxy

Note: There is no httpd process running on this host.

# rpm -qa httpd

2. Configure haproxy

[root@master ~]# vi /etc/haproxy/haproxy.cfg

do not modify global and default setting,

Directly go to 'frontend main' section

Here, change the port where you want your Load Balancer to run.

I will be using port 8080

I will be disabling firewall and selinux for this lab.

frontend main

    bind *:8080

go all the way down to section called 'backend app,

In this section, you will be adding all web server information.

backend app

    balance     roundrobin

    server app1 w1 192.168.10.51:80 check

    server app2 w2 192.168.10.52:80 check

3. Once config is changed, start the service

# systemctl start haproxy

# systemctl enable haproxy

# systemctl status haproxy

4. Now, go to your web server machines. 

a. In my case, its worker node 1 and node2

Install web server and start the service

# yum install httpd

# systemctl start httpd

# systemctl status httpd

# systemctl enable httpd

# systemctl stop firewalld

b. Create a index file

[root@worker1 html]# cat index.html

This is worker node1

[root@worker2 html]# cat index.html

This is Worker node2

5. Now, get the IP of your load balancer server. 

http://192.168.10.50:8080

You should be able to see the web site. if you refresh, you will see new page.

This proves that load balancer is working.

--------------------------------------------------------------

Until now, we configure haproxy manually, lets start configuring haproxy using ansible

1. Lets configure our inventory file as follows,

# ansible --version

# more /etc/ansible/ansible.conf

# cat myhosts

[mylb]

master  ansible_user=root ansible_ssh_pass=changeme ansible_connection=ssh

[myweb]

worker1 ansible_user=root ansible_ssh_pass=changeme ansible_connection=ssh

worker2 ansible_user=root ansible_ssh_pass=changeme ansible_connection=ssh

Note: There is one option available in inventory to group them. and give them group name say web or load balancer.

 

2. Lets automate everything using ansible. Here is the yaml file.

[root@master wk6]# cat mylb.yaml

- hosts: myweb  # myweb comes from inventory file

  tasks:

  - package:

      name: "httpd"

  - copy:

      dest: "/var/www/html/index.html"

      content: " Testing Load Balancer on RHEL7/Centos7"

  - service:

      name: "httpd"

      state: restarted

  - service:

      name: "firewalld"

      state: stopped

      enabled: False

- hosts: mylb

  tasks:

  - name: "Install LB software"

    package:

      name: "haproxy"

  - template:

      dest: "/etc/haproxy/haproxy.cfg"

      src: "haproxy.cfg"

  - service:

      name: "haproxy"

      state: restarted

3. Lets look at the config file for haproxy

Do not modify global and default values.

[root@master wk6]# cat haproxy.cfg

#---------------------------------------------------------------------

# Example configuration for a possible web application.  See the

# full configuration options online.

#

#   https://www.haproxy.org/download/1.8/doc/configuration.txt

#

#---------------------------------------------------------------------

#---------------------------------------------------------------------

# Global settings

#---------------------------------------------------------------------

global

    # to have these messages end up in /var/log/haproxy.log you will

    # need to:

    #

    # 1) configure syslog to accept network log events.  This is done

    #    by adding the '-r' option to the SYSLOGD_OPTIONS in

    #    /etc/sysconfig/syslog

    #

    # 2) configure local2 events to go to the /var/log/haproxy.log

    #   file. A line like the following can be added to

    #   /etc/sysconfig/syslog

    #

    #    local2.*                       /var/log/haproxy.log

    #

    log         127.0.0.1 local2

    chroot      /var/lib/haproxy

    pidfile     /var/run/haproxy.pid

    maxconn     4000

    user        haproxy

    group       haproxy

    daemon

    # turn on stats unix socket

    stats socket /var/lib/haproxy/stats

    # utilize system-wide crypto-policies

    ssl-default-bind-ciphers PROFILE=SYSTEM

    ssl-default-server-ciphers PROFILE=SYSTEM

#---------------------------------------------------------------------

# common defaults that all the 'listen' and 'backend' sections will

# use if not designated in their block

#---------------------------------------------------------------------

defaults

    mode                    http

    log                     global

    option                  httplog

    option                  dontlognull

    option http-server-close

    option forwardfor       except 127.0.0.0/8

    option                  redispatch

    retries                 3

    timeout http-request    10s

    timeout queue           1m

    timeout connect         10s

    timeout client          1m

    timeout server          1m

    timeout http-keep-alive 10s

    timeout check           10s

    maxconn                 3000

#---------------------------------------------------------------------

# main frontend which proxys to the backends

#---------------------------------------------------------------------

frontend main

    bind *:8080  # This is a port where LB will be listening

    #bind *:5000

    acl url_static       path_beg       -i /static /images /javascript /stylesheets

    acl url_static       path_end       -i .jpg .gif .png .css .js

    use_backend static          if url_static

    default_backend             app

#---------------------------------------------------------------------

# static backend for serving up images, stylesheets and such

#---------------------------------------------------------------------

backend static

    balance     roundrobin

    server      static 127.0.0.1:4331 check

#---------------------------------------------------------------------

# round robin balancing between the various backends

#---------------------------------------------------------------------

backend app  # app value can be anything

    balance     roundrobin

    #server  app1 127.0.0.1:5001 check

    #server  app2 127.0.0.1:5002 check

    #server  app3 127.0.0.1:5003 check

    #server  app4 127.0.0.1:5004 check

    #server app1 w1 192.168.10.51:80 check

    #server app2 w2 192.168.10.52:80 check

{% for i in groups[ 'myweb' ] %}

   server app{{ loop.index }} {{ i }}:80 check

{% endfor %}

4. Lets run your playbook

[root@master wk6]# ansible-playbook mylb.yaml

5. Lets verify the content of haproxy.conf file

# cat /etc/haproxy/haproxy.conf

6. Go to the browser with ip of proxy server which is .50

http://192.168.10.50:8080/

You should be able to see the page.

Now, modify the content of one of the indexfile from web server and refresh the LB server, you will find the new pages.

Setup and Configure load balancer and proxy using HAProxy

Configure LB and proxy using haproxy

Requirement:
1. One server for load balancer
2. One or two servers for web servers

In my example I have three servers
Load Balancer: master  - 192.168.10.50
Web servers: worker1, worker2 - 192.168.10.51/52

1. On master server, Install haproxy - comes on RedHat DVD
# yum install haproxy
Note: There is no httpd process running on this host.
# rpm -qa httpd

2. Configure haproxy
[root@master ~]# vi /etc/haproxy/haproxy.cfg
do not modify global and default setting,
Directly go to 'frontend main' section
Here, change the port where you want your Load Balancer to run.
I will be using port 8080
I will be disabling firewall and selinux for this lab.
frontend main
    bind *:8080
go all the way down to section called 'backend app,
In this section, you will be adding all web server information.
backend app
    balance     roundrobin
    server w1 192.168.10.51:80 check
    server w2 192.168.10.52:80 check

3. Once config is changed, start the service
# systemctl start haproxy
# systemctl enable haproxy
# systemctl status haproxy

4. Now, go to your web server machines. 
a. In my case, its worker node 1 and node2
Install web server and start the service
# yum install httpd
# systemctl start httpd
# systemctl status httpd
# systemctl enable httpd
# systemctl stop firewalld

b. Create a index file
[root@worker1 html]# cat index.html
This is worker node1
[root@worker2 html]# cat index.html
This is Worker node2

5. Now, get the IP of your load balancer server. 
http://192.168.10.50:8080

You should be able to see the web site. if you refresh, you will see new page.
This proves that load balancer is working.