Friday, March 27, 2020

Splunk - Install splunk on linux


These steps are followed based on tutorial available on Linkedin for SPLUNK set up

1. Install virtual box -
https://www.virtualbox.org/wiki/Downloads

2. Copy the config file on any directory and run the command on your windows machine

a. Go to that directory and copy the file.
C:\KB\Splunk\2. Getting Started\2.1 Ex_Files_Learning_Splunk.zip\Exercise Files\Vagrantfile
to
E:\Virtual Machines\splunk
This file is simply ruby script file which are Vagrant specific configuration options to configure VMs.
It has all VMs and defined how they interact.
b. Open up powershell and go to the directory and run vagrant up command
> cd E:\Virtual Machines\splunk
> vagrant up

It will initiate an installation process. It will take a while to install everything.


3.  Install Vagrant - (Vagrant is a simply way to  manage virtual machines)
https://www.vagrantup.com/downloads.html

4. Now, open Powershell and cd to E:\Virtual Machines\splunk where you copied the vagrant file. and Run vagrant up command.

> vagrant up

It will install all required VMs on your virtual box.

Note: If you get an error about hyper visor on ur windows machine, go to start menu and type "turn win"
you will see Turn windows features on or off. click on that and go down and find Hyper-v, if its check, uncheck.

sometime, you may have to go to BIOS and disable it.

5. What is splunk?
It is an enterprise tool to manage data.
It is a single place to collect data.
It is used for system logs and machine data.
It collects all your data in one place and and helps you to search, generate report and helps you to visualize your data.
You can also use SPLUNK to triger alerts based on your data..
It comes with free version and Enterprise version.

6. By now, vigrant boxes are booting. You should be able to login and install splunk.
a. Now, download the Splunk.
Go to splunk and go to download.
When click on download link, you will see useful Tools on right side on splunk page. Click on Download via command link (wget). click on link and copy the link.

Download the OS type file. I am downloading for Linux.
https://www.splunk.com/en_us/download/splunk-enterprise.html#tabs/linux
wget -O splunk-8.0.2.1-f002026bad55-linux-2.6-amd64.deb 'https://www.splunk.com/bin/splunk/DownloadActivityServlet?architecture=x86_64&platform=linux&version=8.0.2.1&product=splunk&filename=splunk-8.0.2.1-f002026bad55-linux-2.6-amd64.deb&wget=true'

Note: download the free version (Splunk Free) not the free trial enterprise edition.
You have to create an account to download.

b. Go to powershell window. You should cd to vagrant file location and issue a command vagrant ssh splunk to login to splunk VM.

PS E:\Virtual Machines\splunk> vagrant ssh splunk
vagrant@127.0.0.1: Permission denied (publickey).

Got an error: Permission denued (publickey)

PS C:\WINDOWS\system32> vagrant ssh splunk
A Vagrant environment or target machine is required to run this
command. Run `vagrant init` to create a new Vagrant environment. Or,
get an ID of a target machine from `vagrant global-status` to run
this command on. A final option is to change to a directory with a
Vagrantfile and to try again.
PS C:\WINDOWS\system32> vagrant.exe global-status
id       name   provider   state   directory
-----------------------------------------------------------------------
820d9e5  splunk virtualbox running E:/Virtual Machines/splunk

c. Find the error with debug option, what is really going on.
PS E:\Virtual Machines\splunk> vagrant ssh splunk --debug
.............................................................

DEBUG safe_exec: Command: `"C:\\WINDOWS\\System32\\OpenSSH\\/ssh.EXE"` Args: `["vagrant@127.0.0.1", "-p", "2222", "-o", "LogLevel=FATAL", "-o", "Compression=yes", "-o", "DSAAuthentication=yes", "-o", "IdentitiesOnly=yes", "-o", "StrictHostKeyChecking=no", "-o", "UserKnownHostsFile=/dev/null", "-i", "E:/VirtualMachines/splunk/.vagrant/machines/splunk/virtualbox/private_key"]
vagrant@127.0.0.1: Permission denied (publickey).


d. Renamed this file: E:/Virtual Machines/splunk/.vagrant/machines/splunk/virtualbox/private_key to 111private_key_222

e. Now, login to splunk

PS E:\Virtual Machines\splunk> vagrant ssh splunk
Welcome to Ubuntu 16.04.6 LTS (GNU/Linux 4.4.0-176-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

 * Latest Kubernetes 1.18 beta is now available for your laptop, NUC, cloud
   instance or Raspberry Pi, with automatic updates to the final GA release.

     sudo snap install microk8s --channel=1.18/beta --classic

 * Multipass 1.1 adds proxy support for developers behind enterprise
   firewalls. Rapid prototyping for cloud operations just got easier.

     https://multipass.run/

0 packages can be updated.
0 updates are security updates.

New release '18.04.4 LTS' available.
Run 'do-release-upgrade' to upgrade to it.


vagrant@splunk:~$

Become root user
vagrant@splunk:~$ sudo su -
root@splunk:~#

7. Install Splunk
Note: Do not install splunk as a root user at work place. Its my lab, so I am fine.
Now, at the prompt, paste the copied link.
# wget -O splunk-8.0.2.1-f002026bad55-linux-2.6-amd64.deb 'https://www.splunk.com/bin/splunk/DownloadActivityServlet?architecture=x86_64&platform=linux&version=8.0.2.1&product=splunk&filename=splunk-8.0.2.1-f002026bad55-linux-2.6-amd64.deb&wget=true'

splunk-8.0.2.1-f002026bad55-l 100%[=================================================>] 374.11M  5.86MB/s

It will download the file on current directory

a. Install the package
root@splunk:~# dpkg -i splunk-8.0.2.1-f002026bad55-linux-2.6-amd64.deb

press q and y to accept the license and continue.

It may prompt you for password. this is the password for your splunk.
at the end of the installation, it will give you the address path with port.

if it didn't prompt you for password, you have to go throug splunk start/stop command.

go to bin directory and run splunk start command. (use flag --accept-license)
root@splunk:/opt/splunk/bin# cd /opt/splunk/bin
root@splunk:/opt/splunk/bin# ./splunk start --accept

type q and y to accept the license.

it will propmpt you to enter admin username:


Please enter an administrator username: myadmin
Please confirm new password:


to start splunk at boot run the command
root@splunk:/opt/splunk/bin# ./splunk enable boot-start -user myadmin
root@splunk:/opt/splunk/bin# systemct enable splunk
root@splunk:/opt/splunk/bin# systemctl start splunk
root@splunk:/opt/splunk/bin# ./splunk star



Now, go to web address on the browser and use the IP and port to login.

8. Shutdown Vagrant manage VMs

PS E:\Virtual Machines\splunk> vagrant halt

=====================================

If you don't want to go through all this hassle, use centos.

Download the splunk package
# wget -O splunk-8.0.2.1-f002026bad55-linux-2.6-x86_64.rpm 'https://www.splunk.com/bin/splunk/DownloadActivityServlet?architecture=x86_64&platform=linux&version=8.0.2.1&product=splunk&filename=splunk-8.0.2.1-f002026bad55-linux-2.6-x86_64.rpm&wget=true'

# rpm -ivh splunk-8.0.2.1-f002026bad55-linux-2.6-x86_64.rpm
# cd /opt/splunk/bin
# ./splunk start -accept
# ./splunk start

You will get IP:PORT, paste on browser, use the user/pw to login.

Enjoy !!!

Friday, March 13, 2020

RHEL7 - How to join Redhat machine to Windows Active directory

Join RHEL7 System to Active directory domain

1. Install realm and sssd packages
# yum install reammd sssd

# rpm -qa | egrep "realmd|sssd"

2. Join the A/D domain
# realm join ad.it.business.com -U svc-aduser
Enter the pw:

3. Edit sssd.conf file and change use fully qualified names to False
so that you don't have to type @ad.it.business.com

# vi /etc/sssd/sssd.conf
use_fully_qualified_names = False

4. Permit user to login
# realm permit bhusal


If you want to install GUI
# yum grouplist
# yum groupinstall "Server with GUI"
# shutdown -r now

Monday, March 2, 2020

RHEL - Creating 10TB filesystem on rhel7

Note:
Something to know ...

Rescan the SCSI bus
# echo "- - -" > /sys/class/scsi_host/host0/scan

To expand the same LUN, perform the following task
# echo 1 > /sys/class/scsi_device/device/rescan

Verify
# fdisk -l



$ sudo fdisk -l
WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.

Disk /dev/sda: 268.4 GB, 268435456000 bytes, 524288000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: gpt
Disk identifier: EA4D9CB5-6A7C-417B-90B0-BF71AF7BB69F

#         Start          End    Size  Type            Name
1         2048         4095      1M  BIOS boot
2         4096      2101247      1G  Microsoft basic
3      2101248    524285951    249G  Linux LVM

Disk /dev/sdb: 10995.1 GB, 10995116277760 bytes, 21474836480 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/vg0-root: 37.6 GB, 37580963840 bytes, 73400320 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/vg0-swap: 4160 MB, 4160749568 bytes, 8126464 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/vg0-01: 5368 MB, 5368709120 bytes, 10485760 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/vg0-00: 220.2 GB, 220242903040 bytes, 430161920 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

$ sudo parted /dev/sdb
GNU Parted 3.1
Using /dev/sdb
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) mklabel gpt
(parted) help
  align-check TYPE N                       check partition N for TYPE(min|opt) alignment
  help [COMMAND]                           print general help, or help on COMMAND
  mklabel,mktable LABEL-TYPE               create a new disklabel (partition table)
  mkpart PART-TYPE [FS-TYPE] START END     make a partition
  name NUMBER NAME                         name partition NUMBER as NAME
  print [devices|free|list,all|NUMBER]     display the partition table, available devices, free space, all found partitions, or a
        particular partition
  quit                                     exit program
  rescue START END                         rescue a lost partition near START and END

  resizepart NUMBER END                    resize partition NUMBER
  rm NUMBER                                delete partition NUMBER
  select DEVICE                            choose the device to edit
  disk_set FLAG STATE                      change the FLAG on selected device
  disk_toggle [FLAG]                       toggle the state of FLAG on selected device
  set NUMBER FLAG STATE                    change the FLAG on partition NUMBER
  toggle [NUMBER [FLAG]]                   toggle the state of FLAG on partition NUMBER
  unit UNIT                                set the default unit to UNIT
  version                                  display the version number and copyright information of GNU Parted
(parted) unit TB
(parted) mkpart primary 0 0
(parted) print
Model: VMware Virtual disk (scsi)
Disk /dev/sdb: 11.0TB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name     Flags
1      0.00TB  0.00TB  0.00TB               primary

(parted) mkpart primary 0 11
Warning: You requested a partition from 0.00TB to 11.0TB (sectors 0..21474836479).
The closest location we can manage is 0.00TB to 0.00TB (sectors 34..2047).
Is this still acceptable to you?
Yes/No? n
(parted) p
Model: VMware Virtual disk (scsi)
Disk /dev/sdb: 11.0TB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name     Flags
1      0.00TB  0.00TB  0.00TB               primary

(parted) help
  align-check TYPE N                       check partition N for TYPE(min|opt) alignment
  help [COMMAND]                           print general help, or help on COMMAND
  mklabel,mktable LABEL-TYPE               create a new disklabel (partition table)
  mkpart PART-TYPE [FS-TYPE] START END     make a partition
  name NUMBER NAME                         name partition NUMBER as NAME
  print [devices|free|list,all|NUMBER]     display the partition table, available devices, free space, all found partitions, or a
        particular partition
  quit                                     exit program
  rescue START END                         rescue a lost partition near START and END

  resizepart NUMBER END                    resize partition NUMBER
  rm NUMBER                                delete partition NUMBER
  select DEVICE                            choose the device to edit
  disk_set FLAG STATE                      change the FLAG on selected device
  disk_toggle [FLAG]                       toggle the state of FLAG on selected device
  set NUMBER FLAG STATE                    change the FLAG on partition NUMBER
  toggle [NUMBER [FLAG]]                   toggle the state of FLAG on partition NUMBER
  unit UNIT                                set the default unit to UNIT
  version                                  display the version number and copyright information of GNU Parted
(parted) rm 1
(parted) p
Model: VMware Virtual disk (scsi)
Disk /dev/sdb: 11.0TB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start  End  Size  File system  Name  Flags

(parted) mkpart primary 0 11
(parted) p
Model: VMware Virtual disk (scsi)
Disk /dev/sdb: 11.0TB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name     Flags
1      0.00TB  11.0TB  11.0TB               primary

(parted) q
Information: You may need to update /etc/fstab.

$ sudo parted /dev/sdb
GNU Parted 3.1
Using /dev/sdb
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) p
Model: VMware Virtual disk (scsi)
Disk /dev/sdb: 11.0TB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name     Flags
1      1049kB  11.0TB  11.0TB               primary

(parted) q

$ sudo pvcreate /dev/sdb1
  Physical volume "/dev/sdb1" successfully created.
$ sudo vgcreate datavg /dev/sdb1
sudo vgs  Volume group "datavg" successfully created
$ sudo vgs
  VG     #PV #LV #SN Attr   VSize    VFree
  datavg   1   0   0 wz--n-  <10.00t <10.00t
  vg0      1   4   0 wz--n- <249.00g   4.00m
$ sudo lvcreate -n nesslv -L +99G datavg
  Logical volume "nesslv" created.
$ sudo mkfs.xfs /dev/datavg/nesslv
$ sudo lvremove /dev/datavg/nesslvsudo lvremove /dev/datavg/nesslv
$ sudo lvremove /dev/datavg/nesslv
Do you really want to remove active logical volume datavg/nesslv? [y/n]: y
  Logical volume "nesslv" successfully removed
$ sudo lvcreate -n nesslv -L +10T datavg
  Volume group "datavg" has insufficient free space (2621439 extents): 2621440 required.
$ sudo lvcreate -n nesslv -L +9.9T datavg
  Rounding up size to full physical extent 9.90 TiB
  Logical volume "nesslv" created.
$ sudo mkfs.xfs /dev/datavg/nesslv
meta-data=/dev/datavg/nesslv     isize=512    agcount=10, agsize=268435455 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=0, sparse=0
data     =                       bsize=4096   blocks=2657511424, imaxpct=5
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal log           bsize=4096   blocks=521728, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
$ sudo vi /etc/fstab
$ sudo mkdir /1opt1
$ df -h
Filesystem            Size  Used Avail Use% Mounted on
devtmpfs               32G     0   32G   0% /dev
tmpfs                  32G     0   32G   0% /dev/shm
tmpfs                  32G   20M   32G   1% /run
tmpfs                  32G     0   32G   0% /sys/fs/cgroup
/dev/mapper/vg0-root   35G  1.5G   34G   5% /
/dev/mapper/vg0-01    5.0G  100M  4.9G   2% /var/log
/dev/mapper/vg0-00    206G  3.1G  202G   2% /opt
/dev/sda2            1014M  154M  861M  16% /boot
tmpfs                 6.3G     0  6.3G   0% /run/user/1001
$ sudo vgs
  VG     #PV #LV #SN Attr   VSize    VFree
  datavg   1   1   0 wz--n-  <10.00t 102.39g
  vg0      1   4   0 wz--n- <249.00g   4.00m
$ sudo vi /etc/fstab
$ ls /opt
sc
$ cd /opt
$ sudo du -sh *
3.1G    sc
$ more /etc/fstab

#
# /etc/fstab
# Created by anaconda on Tue Feb 11 05:56:09 2020
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/vg0-root    /                       xfs     defaults        0 0
UUID=31657a30-d845-4032-8f9b-b48fc75f23ed /boot                   xfs     defaults        0 0
##/dev/mapper/vg0-00      /1opt1                    xfs     defaults        0 0
/dev/mapper/vg0-01      /var/log                xfs     nosuid,noexec,nodev 0 0
/dev/mapper/vg0-swap    swap                    swap    defaults        0 0
####
/dev/mapper/vg0-00      /1opt1          xfs     defaults        0 0
/dev/datavg/nesslv      /opt            xfs     defaults        0 0
$ sudo reboot
Connection to 10.85.98.40 closed by remote host.
Connection to 10.85.98.40 closed.
                                                                                                                                v
---------------------------------------------------------------------------------------------------------------------------------

$ ssh admin@192.168..10
$ df -h
Filesystem                 Size  Used Avail Use% Mounted on
devtmpfs                    32G     0   32G   0% /dev
tmpfs                       32G     0   32G   0% /dev/shm
tmpfs                       32G   20M   32G   1% /run
tmpfs                       32G     0   32G   0% /sys/fs/cgroup
/dev/mapper/vg0-root        35G  1.6G   34G   5% /
/dev/mapper/vg0-01         5.0G   92M  5.0G   2% /var/log
/dev/sda2                 1014M  141M  874M  14% /boot
/dev/mapper/datavg-nesslv  9.9T   33M  9.9T   1% /opt
/dev/mapper/vg0-00         206G  3.1G  202G   2% /1opt1
tmpfs                      6.3G     0  6.3G   0% /run/user/1001
$ cd /opt
$ ls
$ cd ..
$ cd /1opt1
$ ls
sc
$ tar -cf /opt/sc.tar sc
tar: /opt/sc.tar: Cannot open: Permission denied
tar: Error is not recoverable: exiting now
$ sudo tar -cf /opt/sc.tar sc
$ du -sh /opt/sc.tar
2.9G    /opt/sc.tar
$ cd /opt
$ ls
sc.tar
$ tar -xvf


#!/bin/bash

if [ ! "$(whoami)" == "chante" ]; then
  echo "Error: script must be run as the chante user"
  exit 1
fi