Sunday, August 5, 2018

Shell Accounts


Main Config files
-----------------

/etc/passwd
format:  
::::::

for second field:
x - restricts user to access the system (shadow password format method), this means that the user has an encrypted password
!! - no password has been set

blank - requires only user name to log on
/etc/shadow
format:   username:encrypted password:last:may:must:warn:expire:disable:reserved

1. User name : It is your login name
2. Password: It your encrypted password. The password should be minimum 6-8 characters long including special characters/digits
3. Last password change (lastchanged): Days since Jan 1, 1970 that password was last changed
4. Minimum: The minimum number of days required between password changes i.e. the number of days left before the user is allowed to change his/her password
5. Maximum: The maximum number of days the password is valid (after that user is forced to change his/her password)
6. Warn : The number of days before password is to expire that user is warned that his/her password must be changed
7. Inactive : The number of days after password expires that account is disabled
8. Expire : days since Jan 1, 1970 that account is disabled i.e. an absolute date specifying when the login may no longer be used
/etc/gshadow
format:

group_name:password:admin:member
/etc/default/useradd
default settings for account creation
/etc/skel
location of default files that will be copied to the user's home directory
/etc/group
location of available groups in the system
format:

group_name:group_password:GID:list_of_members

Password Policy
---------------

Config files:

/etc/pam.d/system-auth
/etc/login.defs
/etc/default/useradd

/etc/pam.d/system-auth
- uses PAM mechanism (Pluggable Authentication Module)
- used in enforcing stronger passwords
- you can see more info by hitting "man pam_unix"
- NOTE: changing this file may break your logins (e.g SSH connections no longer work, your connection is being closed, etc) so be sure to
               have a backup of this file before making any changes
format:
module_interface control_flag module_name module_arguments
** module interfaces **
auth -> requests and verifies the validity of a password
account -> checks if a user account has expired or if a user is allowed to log in at a particular time of day
password -> used for changing user passwords
session -> manages user sessions (e.g like mounting a user's home directory and making the user's mailbox available)

** control flags **
required -> result must be successful for authentication to continue (user not notified)
requisite -> result must be successful for authentication to continue (user notified)
sufficient -> result is ignored if it fails
optional -> result is ignored
include -> can be removed from config

** module names **
- example of module names are: pam_unix.so, pam_deny.so, pam_cracklib.so
pam_unix.so -> used in RHEL 5/6
pam_cracklib.so -> checks the password against the dictionary and other constraints

** module arguments **
retry=3 -> denotes 3 chances
SHA512 -> password hashing algorithm used
minlen=8 -> minmimun password length (supersedes PASS_MIN_LEN in /etc/login.defs)
ucredit=-1 -> atleast 1 uppercase character
lcredeit=-1 -> atleast 1 lowercase character
ocredit=-1 -> atleast 1 special character
dcredit=-1 -> atleast 1 digit

examples:

line that shows password history        password sufficient /lib/security/$ISA/pam_unix.so nullok use_authtok md5 shadow remember=10
line that shows max retries        account required /lib/security/$ISA/pam_tally.so deny=5 no_magic_root reset
shows a strong password policy        password  requisite  /lib/security/$ISA/pam_cracklib.so retry=3 minlen=8 lcredit=-1 ucredit=-1 dcredit=-1 ocredit=-1
/etc/login.defs
- config file for login programs
- config file for "Shadow Suite"
- doesn't use PAM
- this file is required to be present
- see more info by hitting "man login.defs"

#
#       PASS_MAX_DAYS   Maximum number of days a password may be used.
#       PASS_MIN_DAYS   Minimum number of days allowed between password changes.
#       PASS_MIN_LEN    Minimum acceptable password length.
#       PASS_WARN_AGE   Number of days warning given before a password expires.
#
PASS_MAX_DAYS   90
PASS_MIN_DAYS   0
PASS_MIN_LEN    8
PASS_WARN_AGE   7
/etc/default/useradd
- config file uses by "useradd" to get the default settings for the newly created user
- you can also use "useradd -D" to display the config
- see mroe info by hitting "man useradd"

GROUP=100                      # user's initial login group (group must exist)
HOME=/home                     # location of user's home directory
INACTIVE=-1                    # number of days after a password expires until the account is permanently disabled
                               # (0 disables account as soon as password expires, -1 disables the feature)
EXPIRE=                        # # of days on which the account will be disabled (empty string is no expiry date)
SHELL=/bin/bash                # user's shell to use
SKEL=/etc/skel                 # skel directory
CREATE_MAIL_SPOOL=yes          # self explanatory

Managing Shell Accounts
-----------------------

Listing
## checks for empty passwords
awk -F: '($2 == "") {print}' /etc/shadow
Adding
## adds a user account to the system
useradd [option/s]
-c <"comment">  -->enables you to insert a comment
-d   -->specifies home directory
-u -->specifies user id (UID)
-g   -->specifies the primary group by either group id or group name (or -g )
-G   -->specifies the supplementary groups
-k   -->specifies the other directory other than /etc/skel
-m  -->creates user home directory if it doesn't exist
-s   -->specifies login shell
-p   -->specifies user password
-D  -->lists default options for account creation
-Db   -->changes default home directory
-Dg   -->changes default startup group
-Ds   -->changes default startup shell
-DE   -->adds an account expiration date
-Df   -->disables an account for a specified number of days after a password expires
-Ds
  -->change default shell for account creation
-n  -->avoids creating user with their own group (bob:bob)
## prevents creating usernames with their own groups
useradd -n bob

Deleting
## deletes a user account
userdel [options]
-r  -->deletes a user account and user home directory
Updating
## updates and creates users in batch
newusers


## directly edits password file
vipw


UIDs
----

UID/GID
        RID
        Purpose
        Examples
System Users
        0-499
System user accounts
        root, apache, mysql
System Groups
        0-499        
System group accounts
        lp
Normal Users
        500-999        
User accounts outside of LDAP
        devel

Managing Passwords
------------------

General Commands
passwd [options]   ##sets user password

  * if shadowing is enables, there might be a need to manually add the user in
    /etc/shadow and put !! in the password field before issuing passwd command

  -S -->checks status of user account (locked/unlocked)
  -l -->locks a user account
  -u -->unlocks a user account
  -uf -->force unlocks a user account
  -d -->removes a password (changes to passwordless acount)
  -x number_of_days --> sets maximum number of days for which password remains valid
  -n number_of_days --> sets minimum number of days for which password remains valid
  -w number_of_days --> sets the number of days prior to password expiry that the user begins to receive warnings
  -f  --> similar to chage -d 0

usermod [options]   ##modifies use accounts
-l new_user_name user_name        changes username
-c "comment" user_name        changes comment
-d new_home_dir user_name        changes home directory
-s new_shell user_name        changes home directory
-u new_UID user_name        changes UID *only files UID inside the home directory will be changed
-G current_group,group_to_add1,..,group_to_addN user_name  ##add groups to an account
-g new_group_name user_name          ##changes group
-L user_name  ##locks a user
-U user_name  ##unlocks a user

chage [option]   ##sets password expiration date
-d mm/dd/yyyy -->indicates the last date that the password was changed (same with "chage -d YYYY-MM-DD ")
-M max_of_days -->specifies the number of days for which the password is valid
-m min_of_days -->specifies the minimum number of days before the user can change the password
-E mm/dd/yyyy -->specifies the date on which the account will expire
-I number_of_days -->specifies the date on which the account will expire
-W number_of_warning_days -->specifies the number of days before a password expires and warns for expiration
-l -->lists the number of days before an account expires
-d 0 -->adds a prompt at login asking user to change password
example chage outputs:
RHEL 3.X/4.X:
[root@server~]# chage -l bob
Minimum:        0
Maximum:        99999
Warning:        7
Inactive:       -1
Last Change:            Jan 27, 2014
Password Expires:       Never
Password Inactive:      Never
Account Expires:        Never
[root@server~]#

RHEL 5.X/6.X:
[root@server~]# chage -l bob
Last password change                                    : Mar 14, 2015
Password expires                                        : never
Password inactive                                       : never
Account expires                                         : never
Minimum number of days between password change          : 0
Maximum number of days between password change          : 99999
Number of days of warning before password expires       : 7
[root@server~]#

min # of days between password change -> if this is set to 5, you are only allowed to change your password once every 5 days
max # of days between password change -> this is the number of days your current password is valid
warning -> prior number of days system will warn you that your password will expire soon
inactive -> after a password has expired, system will track how many days your account will be inactive until you change your password. when you reach that "inactive" # of days and you haven't change your password, your account will be lock.

Listing
to check if user is existing:
id
grep -i /etc/passwd
 
lastb  ##list unsuccesful login attempts
faillog  ##reports on failed login attempts
lastlog  ##list all recent user login
groups   ##lists groups where the user belongs
 
pwck  ##verifies integrity of password files
grpck  ##verifies integrity of group files
##checks and reset locked account
pam_tally --user bob
pam_tally --reset --user bob
pam_tally2 --> for RHEL 6.4
Changing
##alternative methods of changing user password
echo 'password' | passwd --stdin
echo "user_name:password" | chpasswd
Modifying
faillog -r -u   ##clears fail logins
 
## forces password change upon login
chage -d 0 bob
Removing/Disabling
## disables all password aging
chage -I -1 -m 0 -M 99999 -E -1

Managing Groups
---------------

General Commands
groupadd [options]   ##adds a new group to the system
-g GID -->specifies a numerical id
-r -->creates a system group         

groupmod [options]   ##modifies group information
-g new_GID -->changes current GID
-n new_group_name -->changes current group name
-o -->forces operation

newgrp   ##changes/logins to a new group

gpasswd [option]   ##adds users to groups
-a -->adds a named user to the specified group
-A -->makes the named user the admin of the specified group
-d -->removes the named user from the specified group
-M -->enables you to add several members to the specified group (separated with commas)
-r -->removes the password from the specified group
-R -->prevents users from issuing newgrp comand to join the specified group
Modifying
gpasswd   ##changes group password
Deleting
groupdel   ##deletes a group
grpunconv        ##disables group shadowing
Adding
grpconv          ##enables group shadowing

Logins
------

Listing
# checks last login time for a particular user
lastlog -u
Adding
pwconv  ##enables shadowing
Removing
pwunconv  ##disables shadowing

Switching Accounts
------------------

## switch to another user
su

## switch to another user and makes the shell a login shell
su -c

## switch to root
su -
su - root

## passes a command but not switches to the user
su -c "your_command"

Log Files
---------

/var/log/secure
/var/log/authlog
/var/log/sulog

NIS (Network Information Service)
---------------------------------

/etc/yp.conf
main NIS configuration file
commands
make -C /var/yp  ##updates NIS maps for any changes made to user information (or cd to /var/yp and issue make command)
authconfig-tui  ##configuring system authentication resources (used to bind NIS client to NIS domain)
/usr/lib/yp/ypinit -m  ##builds NIS database on server side

configuring NIS server daemon:
/etc/init.d/ypserv [start|stop|status|restart|condrestart|try-restart|reload|force-reload]
service ypserv [start|stop|status|restart|condrestart|try-restart|reload|force-reload]

configuring NIS passwd daemon (lets users change their passwords on the nis server from nis clients):
/etc/init.d/yppasswdd [start|stop|status|restart|condrestart|try-restart|reload|force-reload]
service yppasswdd [start|stop|status|restart|condrestart|try-restart|reload|force-reload]

configuring NIS map transfer daemon (used to speed up the transfer of very large nis maps):
/etc/init.d/ypxfrd [start|stop|status|restart|condrestart|try-restart|reload|force-reload]
service ypxfrd [start|stop|status|restart|condrestart|try-restart|reload|force-reload]

configuring NIS client daemon:
/etc/init.d/ypbind [start|stop|status|restart|condrestart|try-restart|reload|force-reload]
service ypbind [start|stop|status|restart|condrestart|try-restart|reload|force-reload]

to verify if a user authentication has been updated:
ypmatch passwd
getent passwd
ypcat passwd

ypwhich -m  ##queries NIS database

yppasswd  ##changes user password using a normal account
yppasswd -p ##changes user password using a root account (no need to update the database afterwards)

ypdomainname ##prints NIS domain (or domainanme)

useradd -p   ##to prevent encountering NIS password errors

rpcinfo -u yppasswd  ##another way of checking yppasswdd status

Sudo
----

/etc/sudoers
- main config file
- to edit this file, use "visudo" command (IMPORTANT: don't edit manually)
Format:

  =
   |                                          |
   |__ This can be a user alias               |
                                              |
                This can be a command alias __|

Examples:
bob          ALL=(ALL)    /bin/su                # specifies a single command for a single user
UNIX_ADMINS  ALL=(ALL)    /bin/su                # specifies a single command for a list of users
bob          ALL=(ALL)    NOPASSWD: /bin/su      # bypass password during sudo
UNIX_ADMINS  ALL=(ALL)    NOPASSWD: ALL          # indicates user can run ALL commands
UNIX_ADMINS  ALL=(ALL)    NOPASSWD: UNIX_CMDS    # uses a command alias (list of commands)
Declaring User Aliases:
# User_Alias = , , ...,
User_Alias  UNIX_ADMINS = bob, john, mary

Declaring Command Aliases:
# Cmnd_Alias = , , ...,
Cmnd_Alias  UNIX_CMDS = /sbin/reboot, /sbin/fdisk, /sbin/shutdown
Cmnd_Alias  FILE_CMDS = /bin/chown, /bin/cp
Cmnd_Alias  MISC_CMDS = /bin/chown -R testuser:testuser /home/testuser/B/, /bin/cp /home/testuser/*.dmp /root/dmp_files/

You may add sudo rules under /etc/sudoers.d/. Example is:

[vagrant@node1 ~]$ sudo cat /etc/sudoers.d/vagrant        
%vagrant ALL=(ALL) NOPASSWD: ALL                          
[vagrant@node1 ~]$ 
Displaying
# checks what sudo commands you have
sudo -l
Modifying
# edits sudoers file
visudo

Troubleshooting
---------------

Unable to change NIS password
when error below is encountered, be sure to always add NIS users with "-p" option: useradd -p

[root@rhn1 ~]# yppasswd -p user3
Changing NIS account information for user3 on rhn1.
Please enter root password:
Changing NIS password for user3 on rhn1.
Please enter new password:
Please retype new password:
Error while changing the NIS password.
The NIS password has not been changed on rhn1.
Cannot issue yp commands even though ypbind is running
error messages:
No such map passwd.byname. Reason: Can't bind to server which serves this domain
scenario:
[root@rhn2 ~]# ypcat passwd
No such map passwd.byname. Reason: Can't bind to server which serves this domain
[root@rhn2 ~]#

solution/s:
- try turning off NetworkManager: service NetworkManager stop
- check if domain is correct: ypdomainname
- check connectivity to NIS server
- check entry in yp.conf: domain server
Couldn't chdir to home directory
- try to disable se linux in /etc/sysconfig/selinux: SELINUX=disabled
Internal NIS error
issue "make -C /var/yp" on NIS server
NIS hang
- restart autofs
Wrong time inside chrooted environment
wrong time inside chrooted environment        Scenario:
"ls" shows timestamps which are 5 hours off by the corect time

Resolution:
cp -p /usr/share/zoneinfo/America/New_York /home/jail/etc/localtime
  # where /home/jail/etc/ is the chroot's location of config files
Unable to save sudoers file
"What now?"

--> respond with .. x - exit withought saving the changes
Needs TTY?
sudo: sorry, you must have a tty to run sudo

--> comment out "Defaults  requiretty" in visudo
--> note that this method is very insecure
--> or for a particular user: "Defaults:bob!requiretty"
Unable to change group ownership
colon not permitted

--> you can encounter this on commands which uses colon
--> example is: /bin/chown root:root /dir/
--> use this instead: /bin/chown root.root /dir/
--> then save the changes
This account is currently not available.
--> you get this when doing ssh or su'ing to a specific user (user locked down)
--> this is because it has a shell of /bin/nologin or /bin/false (check in /etc/passwd)
--> change the shell to /bin/bash: chsh -s /bin/bash bob

Tutorials/Tips and Tricks
-------------------------

creating multiple users using script
1. create a text file containing list of users

2. use text file for the script below:
for i in $(cat )
do
useradd -m -d /root/home/$i -s /bin/bash -g $i
done

1. create text file
touch batch-user-add.txt

2. create user list and append the following:
user1:password:1001:513:Student Account:/home/user1:/bin/bash
user2:password:1002:513:Sales user:/home/user2:/bin/bash
user100:password:1100:513:Sales user:/home/user100:/bin/bash
tom:password:1110:501:Guest Account:/home/guest:/bin/menu
jerry:password:1120:501:Guest Account:/home/guest:/bin/menu

3. create users in batch
newusers batch-user-add.txt
format of text file:
loginname:password:uid:gid:comment:home_dir:shell

4. verify that the files below are updated.
less /etc/passwd
less /etc/group
less /etc/shadow
Adding supplementary groups to an account
1. id
# id icmadmin
uid=513(icmadmin) gid=513(icmadmin) groups=513(icmadmin),510(dssdicmp),521(ibmcmgrp)

2. usermod -G ,   *include the existing subgroups
usermod -G dssdicmp,ibmcmgrp,db2admin icmadmin

3. id
# id icmadmin
uid=513(icmadmin) gid=513(icmadmin) groups=513(icmadmin),510(dssdicmp),521(ibmcmgrp),520(db2admin)
Enabling nis services/daemons at boot time
[root@rhn1 nfs]# chkconfig | grep ^yp
ypbind          0:off   1:off   2:off   3:off   4:off   5:off   6:off
yppasswdd       0:off   1:off   2:off   3:off   4:off   5:off   6:off
ypserv          0:off   1:off   2:off   3:off   4:off   5:off   6:off
ypxfrd          0:off   1:off   2:off   3:off   4:off   5:off   6:off
[root@rhn1 nfs]#
[root@rhn1 nfs]# chkconfig ypbind on
[root@rhn1 nfs]# chkconfig yppasswdd on
[root@rhn1 nfs]# chkconfig ypserv on
[root@rhn1 nfs]# chkconfig ypxfrd on
[root@rhn1 nfs]# chkconfig | grep ^yp
ypbind          0:off   1:off   2:on    3:on    4:on    5:on    6:off
yppasswdd       0:off   1:off   2:on    3:on    4:on    5:on    6:off
ypserv          0:off   1:off   2:on    3:on    4:on    5:on    6:off
ypxfrd          0:off   1:off   2:on    3:on    4:on    5:on    6:off
[root@rhn1 nfs]#
Setting up chroot
- chrooting is a good way of containing an isolated environment

Different ways of using chroot command:
chroot /path/to/new/root
chroot /path/to/new/root /path/to/server
chroot [options] /path/to/new/root /path/to/server

Requirements:
1. you need to set your jail directory (e.g /home/jail or /chroot)
2. you need to copy required binaries, libraries, config files etc into your chrooted environment (e.g required libs for /bin/bash must be copied over)
3. you need to copy passwd, group, and shadow files on your chrooted environment
4. you must set a chroot shell for your chrooted users (e.g /bin/chroot-shell)
#!/bin/sh
/usr/bin/sudo /usr/sbin/chroot /home/jail /bin/su - $USER "$@"
Find files owned/not owned by a specific user
find / -user bob
find / ! -user bob
Configuring NIS
[NIS server]

1. install yp utilities and set NIS domain afterwards:
yum install yp*
ypdomainname nisdomain
###to make nisdomain permanent, add this line in /etc/sysconfig/network
NISDOMAIN=nisdomain
2. start yp server daemon: service ypserv start
3. start yp passwd daemon: service yppasswdd start
4. start yp forward daemon: service ypxfrd start
5. initialize NIS domain: /usr/lib/yp/ypinit -m
6. edit config file:
vi /etc/yp.conf
###add this line below
ypserver rhn1
7. restart nis daemons and services:
service ypserv restart
service yppasswdd restart
service ypxfrd restart
8. start NIS bind: service ypbind start
9. enable services at boot time:
chkcofing ypserv on
chkconfig yppasswdd on
chkconfig ypxfrd on
chkconfig ypbind on
10. test NIS: ypcat passwd

[NIS client]

1. install yp tools: yum install yp-tools
2. bind NIS client to NIS server: authconfig-tui
*check Use NIS
*uncheck Local authorization is sufficient
3. test NIS: ypcat passwd
AutoFS configuration for home directories
[NIS/NFS client]

1. install autofs: yum install autofs

2. edit auto master:
vi /etc/auto.master
##add the following line below
/home        /etc/auto.home                --timeout=600

3. create map file:
vi /etc/auto.home
##add the following line below
*        -fstype=nfs                rhn1:/home/&

4. start autofs:
service autof start
chkconfig autofs on

5. test auto fs: su user1
Moving user to a new home directory
This is a simple hack if you want to move an existing user's home directory to a new location. This is helpful if
you have a new partition create for user's home dirs (e.g /home2)

1. Execute this 1 command
usermod -md /home2/bob bob
  # /home2 must exist
  # /home2/bob will be created automatically by the command
  # bob's files on the old home dir will be moved to the new
  # the old home dir (together with its contents) will be deleted

2. Test as root
su - bob
  # this shouldn't return an error

3. You can also see that the /etc/passwd entry has been updated as well

grep bob /etc/passwd


Saturday, August 4, 2018

SSH


Configuration
-------------

- server config file is /etc/ssh/sshd_config
- user config file is ~/.ssh/config

Options and Parameters on /etc/ssh/sshd_config

ClientAliveInterval
number of interval in seconds in checking if an ssh client is still responsive
ClientAliveCountMax
number of times ssh will check if ssh client is still responsive or not

Commands
--------

##forces ipv4 packets
ssh -4

##removes server from known_hosts
ssh-keygen -R

##fixes rsa/dsa keys
ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key
ssh-keygen -t dsa -f /etc/ssh/ssh_host_dsa_key

## reads a key
ssh-keygen -if

## ssh host key
/etc/ssh/ssh_host_key.pub

ssh-keygen -ef ~/.ssh/id_dsa.pub --> converts OpenSSH key to SSH2 key
ssh-keygen -if ~/.ssh/id_dsa_SSH2 --> converts SSH2 key to OpenSSH key


SSH2 vs OpenSSH format

SSH2 key
 ---- BEGIN SSH2 PUBLIC KEY ----
Comment: "2048-bit RSA, converted from OpenSSH by root@server"
AAAAB3NzaC1yc2EAAAABIwAAAQEA04dh1xm3KfeztO9zcrofeuddJ9yQ7RCSrUibD6vw8W
xATGW5kD/a5ZjtF/GURh5AbUcZnjVjIn7RnFbSLeeZb+SuLNREnOHrtAkwcBUY+dSGrqhy
+8Mzfp2oI9UFKP53rCp1ttM1kquB50LozTh3y278Po4YdfwWlb43laTB97qqMFjzx0uK5R
AgvgyVHtGKUY9Q+VVq++cHIfUzYlCMILg+hh+EBSxRVtkYSTxGZgjOKyAjpzjggVgJEi0R
XzFSuVpy5MbtTIIYpyS0A+RN1874TSWsCU9+N6CkDxQBY3fWrHHvjhCXy7Uw5hmHLCUyHe
+c0Gh7kFMHNK6jXH7HEw==
---- END SSH2 PUBLIC KEY ----
OpenSSH key (continious long line of key)
    ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA04dh1xm3KfeztO9zcrofeuddJ9yQ7RCSrUibD6vw8WxATGW5kD/a5ZjtF/GURh5AbUcZnjVjIn7RnFbSLeeZb+SuLNREnOHrtAkwcBUY+dSGrqhy+8Mzfp2oI9UFKP53rCp1ttM1kquB50LozTh3y278Po4YdfwWlb43laTB97qqMFjzx0uK5RAgvgyVHtGKUY9Q+VVq++cHIfUzYlCMILg+hh+EBSxRVtkYSTxGZgjOKyAjpzjggVgJEi0RXzFSuVpy5MbtTIIYpyS0A+RN1874TSWsCU9+N6CkDxQBY3fWrHHvjhCXy7Uw5hmHLCUyHe+c0Gh7kFMHNK6jXH7HEw== root@server

Tutorials
---------

Creating SSH keys
ssh-keygen -f /path/to/private_key -t rsa -N ''
  # specifies output file for private key and disables passphrase
  # public key will be created on same path
Passwordless SSH 1
1. create public and private keys (verify if files are created under ~/.ssh): ssh-keygen -t rsa
2. copy A's public key to B's authorized_keys file (authorized_keys will be automatically created under B's ssh dir):
ssh-copy-id -i /root/.ssh/id_rsa.pub user_name@machine_B
or
scp ~/.ssh/id_rsa.pub user_name@machineB:/user_home_dir_path/.ssh/authorized_keys
3. make sure that ~/.ssh and ~/.ssh/authorized_keys are at 700  and 600 permissions respectively
4. log in to machine B without password
5. repeat steps 1 - 3 inside machine B (same concept).

NOTE:
- target home directory should be of 700 or 750 permissions
- target .ssh directory must be of 700 permission
- files under target .ssh/ directory must be 600
Passwordless SSH 2: (with modifications on local passwd file)
1. create public and private keys (verify if files are created under ~/.ssh): ssh-keygen -t rsa
2. copy A's public key to B's authorized_keys file (authorized_keys will be automatically created under B's ssh dir):
ssh-copy-id -i /root/.ssh/id_rsa.pub user_name@machine_B
or
scp ~/.ssh/id_rsa.pub user_name@machineB:/user_home_dir_path/.ssh/authorized_keys
3. make sure that ~/.ssh and ~/.ssh/authorized_keys are at 700  and 600 permissions respectively
4. check local passwd on target server - second field shouldn't have an "x"

x should be eliminated:
srpadm:x:7016:1003:SAP System Administrator:/usr/sap/SRP:/bin/csh

entry should like be this:
srpadm:5rPrU590FyVIs:7016:1003:SAP System Administrator:/usr/sap/SRP:/bin/csh 

use this command to change local passwd file only and not the NIS passwd file:
passwd -r files

5. log in to machine B without password
6. repeat steps 1 - 3 inside machine B (same concept).

converts ssh key formats
to convert this format into openssh format (for unix/linux based systems):
---- BEGIN SSH2 PUBLIC KEY ----
Comment: "rsa-key-20130424"
AAAAB3NzaC1yc2EAAAABJQAAAIBzxhgzK3bv7yOKXpZR5uIru6OoKANrkn2MmbAC
wSIutaZCKuOaPAPGQv/YPI3JPk0UCOvgtjN/G3Ejm3gzFw/g35rmdQJj8PtPtugE
jgZnWxYWyhsOoAepeGrfw6Wu/tFBHctHt1VnPzRqZU+gwk1dCzuQP46lYyosXLKF
CLZFKQ==
---- END SSH2 PUBLIC KEY ----

execute this command:
ssh-keygen -i -f /tmp/identity2.pub >> ~/.ssh/authorized_keys
Supress banners
1. This will suppress banners
ssh -q my.host.com

2. Annoyed of "Are you sure you want to continue connecting (yes/no)?"? You can use "StrictHostKeyChecking" option to make ssh automatically add the remote hosts's fingerprint (aka public key) into your known_hosts file. This can be useful if you have multiple servers to access and you don't want to manually reply "yes" on each of them.

bash-3.2$ for i in `cat servers`; do ssh -o "StrictHostKeyChecking no" $i "hostname"; done
hostA.com
hostB.com
hostC.com
bash-3.2$
Daily scenarios that are annoying
and can be suppressed
The following can be added under ~/.ssh/config. An example is:

Host *
  StrictHostKeyChecking no
  UserKnownHostsFile /dev/null

1. Disable prompts for host keys.
StrictHostKeyChecking no

2. Disable known_hosts file. This is applicable if you a remote hosts
whose fingerprint constantly changes.
UserKnownHostsFile /dev/null
Generates public key from
private key
ssh-keygen -y -f ~/.ssh/id_rsa > ~/.ssh/id_rsa.pub

Troubleshooting
---------------

Common issues
# example of correct password but unable to login due to PAM
user@remote.server.com's password:
Connection closed by 192.168.54.100 

# example of incorrect password
user@remote.server.com's password:
Permission denied, please try again.
 
# double authentication
error:
[esl-playbooks]_$ ssh user@server.com
Permission denied (publickey).

solution:
double authentication - you need to login via password and public key at the same time

# no matching host key type found. Their offer: ssh-dss
Resolution:
Add this on ssh command: ssh -oHostKeyAlgorithms=+ssh-dss 192.168.1.34

# no matching key exchange method found. Their offer: diffie-hellman-group1-sha1
Resolution:
Add this on ssh command: ssh -oKexAlgorithms=+diffie-hellman-group1-sha1 inv-filerd.msred.dom




Friday, August 3, 2018

Logical Volume Manager (LVM)


Concepts
--------

PHYSICAL VOLUME (PV): physical devices, local disk or luns
|
|
VOLUME GROUP (VG): groups of pvs
|
|
LOGICAL VOLUME (LV): partitions inside vg
|
|
FILESYSTEM (FS): ext2/3/4,DOS,swap,etc..

Paritions
---------

/proc/partitions  ##contains list of known partitions to kernel

Listing/Viewing
fdisk   ##adds partition to a device
fdisk -l  ##lists available devices together with partition tables
fdisk -l | grep sd  ##filters scsi devices
cfdisk   ##similar to fdisk but with less features
 
## to lists partition codes
sfdisk -T
Modifying
## rescans the whole disk (useful in resizing VMDK disks)
echo 1 > /sys/block/sd/device/rescan
pvresize /dev/sdc
 
partprobe   ##force update the /proc/partition file without rebooting (no longer works for rhel6)
partx -a   ##force update the /proc/partition file without rebooting (works for rhel6)
Debugging
dumpe2fs   ##checks deleted partitions
e2fsck   ##checks if partition is readable

Physical Volumes
----------------

examples of PVs:

/dev/sdb
/dev/sdb[12]

Commands:

Displayng
## displays info reg physical volumes
pvdisplay  --> more detailed output
pvs  --> quick view
pvscan
pvdisplay --ignorelockingfailure  --> use incase of r/o filesystem

note: you need a physical device or a partition from a physical device to create a physical volume

here are the rest of pv commands:
[mvineza@secutil07 ~]$ ls /sbin/pv*
/sbin/pvchange  /sbin/pvcreate   /sbin/pvmove    /sbin/pvresize  /sbin/pvscan
/sbin/pvck      /sbin/pvdisplay  /sbin/pvremove  /sbin/pvs
[mvineza@secutil07 ~]$
Creating
## creates and display physical volume/s
pvcreate

## forcely initialize/reinitialize a physical device (for previously used devices)
pvcreate -ff
Removing
## forcely removes a physical volume from a volume group
pvremove
 
Volume Groups
-------------

commands:

Displaying
## displays vg info
vgdisplay
vgdisplay --> for a specific vg
vgdisplay -v   --> list also pvs and lvs associated with it
vgscan --> scan all disks for volume groups and rebuild caches
vgs --> quick view
 
here are list of vg commands:
$ ls /sbin/vg*
/sbin/vgcfgbackup   /sbin/vgck       /sbin/vgdisplay  /sbin/vgimport       /sbin/vgmknodes  /sbin/vgrename  /sbin/vgsplit
/sbin/vgcfgrestore  /sbin/vgconvert  /sbin/vgexport   /sbin/vgimportclone  /sbin/vgreduce   /sbin/vgs
/sbin/vgchange      /sbin/vgcreate   /sbin/vgextend   /sbin/vgmerge        /sbin/vgremove   /sbin/vgscan
$
Creating/Modifying
## creates vg
vgcreate   --> whole pv will be used for vg
vgcreate -c n   --> clustered environment, -c option signifies that vg is accessible by other nodes
vgcreate -s [MGK]   --> indicate the physical extent size
vgcreate -L size   --> if you don’t want to use the entire pv
 
## increase vg
vgextend
 
## activates a volume group
vgchange -a y
Deleting
## decrease vg
vgreduce
vgreduce --removemissing  --> for error similar to these: Couldn't find device with uuid F27gpk-Gpte-11yf-WTvH-94ez-QpnF-fkMmu3

Logical Volumes
---------------

Displaying
## displays info on logical volumes
lvdisplay /dev//
lvscan
lvs
 
here is a list of lv commands:

$ ls /sbin/lv*
/sbin/lvchange   /sbin/lvdisplay  /sbin/lvmchange    /sbin/lvmdump  /sbin/lvmsar    /sbin/lvrename  /sbin/lvscan
/sbin/lvconvert  /sbin/lvextend   /sbin/lvmconf      /sbin/lvmetad  /sbin/lvreduce  /sbin/lvresize
/sbin/lvcreate   /sbin/lvm        /sbin/lvmdiskscan  /sbin/lvmsadc  /sbin/lvremove  /sbin/lvs
$
Creating
## basic way of creating an LV
lvcreate [options]

## if you want to explicitly define its characteristics, use this
lvcreate -n -L [M|G]

## if you want to consume all the space in the volume group
lvcreate -n -l 100%FREE

Resizing
## increases size of a logical volume
lvextend -L +[MKG] /dev//
lvextend -L +[MKG] /dev/mapper/-
lvextend -L [M|G] /full/path/to/lv  --> sets the size of lv instead of adding space (example is with + sign above)
lvextend -L +50g /dev/VG1/LV1 --> for redhat 5.1
lvextend -l +100%FREE /dev/mapper/- --> extends LV to 100% of the free space in the VG

## increasing by pv (uses whole pv)
lvextend /dev/mapper/vg_root-lv_root /dev/sda4

## decreases size of a logical volume
lvreduce -L -[MKG] /dev//
lvreduce -L -[MKG] /dev/mapper/-

Filesystem
----------

commands:

Creating
## creates a filesystem on top of lv
mkfs -t /dev/mapper/-

## other way
mkfs. [options]         
-c     --> checks for bad blocks before the file system is created
-v    --> verbose output
-l      --> obtains a list of currently known bad blocks from the specified file
-t      --> enables you to specify the file system type of the file system to create on the partition
-m   -->sets a percentage space to be reserved for the root user

to formats a device into a specific filesystem:

## creates a swap filesystem
mkswap

## creates an ext2 filesystem
mke2fs -j
Resizing
## resizes a file system
e2fsck -f /dev//  --> checks a linux ext2/3/4 filesystem
resize2fs /dev//  --> resizes an ext2/3/4 filesystem
resize2fs -f /dev//  --> forcefully resizes an ext2/3/4 filesystem (online resizing)
resize_reiserfs /dev//  --> online resizing for suse linux
Modifying
## adds label to a storage device
e2label
tune2fs -L

## prints the label of a storage device
e2label


Troubleshooting
---------------

Missing PV/VG/LV
One possible reason if your LVM device doesn't show up in lvm commands like pvs, lvs, or vgs is that the underlying LUN/storage device got vanished, corrupted, or its LUN assignment has changed.

Example of the error in /var/log/messages is:
Dec 24 19:56:42 orcl01 kernel: sd 2:0:8:0: Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatically remap LUN assignments.

On possible resolution is to scan the LVM device again
# Take backup of your data.
# umount /u01/LocalBackupIBM
# pvscan
# vgscan
# lvscan
# Run vgs & lvs ( If you can't see the volume group then you can proceed with the next step)
# vgcfgrestore -f /etc/lvm/archive/orcl01_backupsIBM_00001-493843090.vg orcl01_backupsIBM
# lvs
Unable to remove LV
Issue:

You see this error when remove LV via `lvremove`
  Logical volume iscsi_vg/iscsi_lv is used by another device.

Resolution:

1. Find out what is holding the LV and take note of the major and minor numbers.
[root@node1 ~]# dmsetup info -c
Name                Maj Min Stat Open Targ Event  UUID
ha_vg-ha_lv         253   3 L--w    0    1      0 LVM-1C5GryiDYDe0lllqq1O8f9HsJyTpoUbe8OgaMOFiV5HEG4bi5fzzXLNHubN6A7pw
VolGroup00-LogVol01 253   1 L--w    2    1      0 LVM-SA8LTUF2yzFEV1RdgThw0ZiRxhyHFKuUIAjIC6ScnMtvH67BTyTN31hd82bgDSzd
VolGroup00-LogVol00 253   0 L--w    1    1      0 LVM-SA8LTUF2yzFEV1RdgThw0ZiRxhyHFKuUj6b8IVKEw37bTwOqy81Ud3juFCSJBg12
iscsi_vg-iscsi_lv   253   2 L--w    1    1      0 LVM-9ZTlpYoUyX9xLbFn2NoTfp3313n89MMh81CFS2y999Cvv9DqQbxdZ9XUnI3xx0QG
[root@node1 ~]#

2. Delete it manually.
[root@node1 ~]# ls -l /sys/dev/block/253\:2
lrwxrwxrwx. 1 root root 0 Jun 15 06:37 /sys/dev/block/253:2 -> ../../devices/virtual/block/dm-2
[root@node1 ~]# dmsetup remove /dev/dm-3                                                                                                                                                                                                     
[root@node1 ~]#

3. You can now remove the LV successfully.
[root@node1 ~]# lvremove /dev/mapper/iscsi_vg-iscsi_lv -f
  Logical volume "iscsi_lv" successfully removed
[root@node1 ~]#
removing a stale device
similar errors you may encounter:
/dev/test_vg/test_lv: read failed after 0 of 4096 at 5368643584: Input/output error
/dev/sdb: read failed after 0 of 4096 at 0: Input/output error

cause:
physical device was remove before deleting from OS

solution:
dmsetup remove /dev/test_vg/*
echo 1 > /sys/block/sdb/device/delete
or
reboot

Tutorials
---------

technique in extending lvs by using physical extents
--> this will prevent you from wasting space in your disk


sample extension using "-L +50G" traditonal human readable size (you can see 1.34G wasted or not used during lv extension)
[root@rt01 /]# pvs
  PV         VG         Fmt  Attr PSize  PFree
  /dev/sda2  VolGroup00 lvm2 a-   50.38G     0
  /dev/sdb   VolGroup00 lvm2 a-   51.97G  1.34G
[root@rt01 /]#

extending using physical extents:
1. determine first if how many remaining free extents you have by extending the lv above the maximum size
[root@rt01 /]# lvextend -L +100g /dev/VolGroup00/lv_root --> here we extend it 100GB which is above 50GB
  Extending logical volume lv_root to 201.00 GB
  Insufficient free space: 3200 extents needed, but only 1642 available --> here you will see 1642 extents available
[root@rt01 /]#

2. use the value in #1 to extend your lv by using "-l" flag instead of "-L"
[root@rt01 /]# lvextend -l +1642 /dev/VolGroup00/lv_root
  Extending logical volume lv_root to 152.31 GB
  Logical volume lv_root successfully resized
[root@rt01 /]#

3. check now your remaining space (you can see no space is wasted)
[root@rt01 /]# pvs
  PV         VG         Fmt  Attr PSize  PFree
  /dev/sda2  VolGroup00 lvm2 a-   50.38G    0
  /dev/sdb   VolGroup00 lvm2 a-   51.97G    0
[root@rt01 /]#
resizing a a single vmdk disk
1. edit virtual disk in vmware settings
2. reboot host
3. create new partition
  # fdisk /dev/sda
  # n > primary > follow defaults
  # t > > 8e
  # w
4. reboot (at this point I don't know if it is possible not to reboot)
5. pvcreate /dev/sda4
6. vgextend vg_root /dev/sda4
7. lvextend /dev/mapper/vg_ftp01-lv_root /dev/sda4
8. resize2fs /dev/mapper/vg_ftp01-lv_root
expanding a non-LVM partition
1. shutdown server
2. add storage
3. insert gparted cd into cdrom
4. boot server from cd
5. use gparted to expand partition

## reminders in designing your partitions

when layouting partitions, make sure to make / the last partition

BAD layout:
/dev/sda1 = /
/dev/sda2 = /boot
/dev/sda3 = swap

--> in this scenario, / is the first partition
--> there is another partition right next to it
--> extending / will be difficult because you must consider the starting and end boundaries of the cylinder

GOOD layout:
/dev/sda1 = /boot
/dev/sda2 = swap
/dev/sda3 = /

--> there must be no partitions after /
--> in that case, you can extend it without worrying about the neighboring cylinder
Destroying an LVM filesystem
1. Remove LV
lvremove

2. Deactivate and remove VG
vgchange -a n
vgremove

3. Remove PV
pvremove
Resizing tmpfs
tmpfs is a RAM based temporary file system which is generally mounted on /dev/shm. To to extend the tmfs file system use below steps:

1. Check initial size: # df -h /dev/shm
2. Edit fstab and specify tmpfs size
  # vi /etc/fstab
  tmpfs                   /dev/shm                tmpfs   size=4g        0 0
3. Remount: # mount -o remount /dev/shm
4. Check final size: # df -h /dev/shm
Moving data to another disk without any downtime
If you want to migrate your data to a new disc - let's say you decided to
purchase another storage appliance and you want your MYSQL datadir to move to
that appliance without any downtime, LVM mirroring is the way to go.

1. Prepare your LUN and assign/zone it to your server

2. Once the LUN is available, add it to the current VG that holds the MYSQL
datadir
# vgextend myql_vg /dev/sdc

3. Create a mirror of the MYSQL logical volume pointing to the newly added LUN
(this is the RIGHT side of the mirror)
# lvconvert -m1 /dev/mapper/mysql_vg-mysql_lv /dev/sdc

4. Wait until the sync is complete (must be 100% under Cpy%Sync field)
# lvs

5. Once the sync is complete, remove/detach the LEFT side of the mirror (that is
the old LUN) from the MYSQL logical volume
# lvconvert -m0 /dev/mapper/mysql_vg-mysql_lv /dev/sdb

6. Remove the old LUN from the MYSQL VG
# vgreduce mysql_vg /dev/sdb

7. Remove PV label
# pvremove /dev/sdb

8. Detach it from your server

9. Hola! You were able to move your MYSQL database to a new and faster disk
without a  downtime!

Renaming a volume group in RHEL/CENTOS 6
1. Verify old name
# vgs

2. Rename VG
# vgrename old_vg_name new_vg_name

3. Backup and replace references in fstab
# cp -p /etc/fstab /etc/fstab.$(date +%F)
# grep old_vg_name /etc/fstab
/dev/mapper/old_vg_name-lv_root    /                       ext4    defaults        1 1
/dev/mapper/old_vg_name-lv_swap    swap                    swap    defaults        0 0

4. Backup and replace referencs in grub.conf
# cp -p /boot/grub/grub.conf /boot/grub/grub.conf.$(date +%F)
# grep old_vg_name /boot/grub/grub.conf
kernel /vmlinuz-2.6.32-642.1.1.el6.x86_64 ro root=/dev/mapper/old_vg_name-lv_root ... rd_LVM_LV=old_vg_name/lv_root ... rd_LVM_LV=old_vg_name/lv_swap ...

5. Backup and rebuild initramfs
# cp /boot/initramfs-$(uname -r).img /boot/initramfs-$(uname -r).img.bak
# dracut -f

6. Restart server
# reboot


Thursday, August 2, 2018

MariaDB Backups


Backup types
------------

1. Logical backups
  - information and records are in plain text
  - DB structure is retrieve by querying the DB
  - slower because server must access DB and convert it into logical format
  - performed while server is online
  - backups don't include log/configuration files

2. Physical backups
  - consist if raw copies of DB directories
  - output is more compact
  - can include config/log files
  - portable only to other machines w/ similar H/W and software
  - faster than logical backups
  - should be performed while server is offline or while tables in all DBs
    are locked to prevent changes during backup

Performing Logical Backups
--------------------------

mysqldmp -u root -p inventory > /backup/inventory.dmp
  -> -p will prompt for password
  -> inventory is the DB to backup
  -> use --all-databases to logically backup all DBs
  -> mysqldump requires atleast SELECT, SHOW VIEW, and TRIGGER privileges
  -> other useful options:
     --add-drop-tables     -- adds DROP TABLE before each CREATE TABLE statement
     --no-data             -- dumps only DB structure, not contents
     --lock-all-tables     -- no new record can be inserted during backup
     --add-drop-databases  -- adds DROP DATABASE before each CREATE DATABASE
                              statement
Performing Physical Backups
---------------------------

Example of tools that is used are ibbackup, cp, mysqlhotcopy, and lvm.
Using LVM keps downtime of DB short. This great when DB is put on
a dedicated partition.

mysqladmin variables | grep datadir # -- checks where MariaDB files are stored

df /var/lib/mysql # -- verifies on what logical volume it resides
vgdisplay vg0 | grep Free # -- prints free space on volume group
systemctl stop mariadb # -- shuts down service
lvcreate -L20G -s -n mariadb-backup /dev/vg0/mariadb  -- creates LVM snapshot
mount /dev/vg0/mariadb-backup /mnt/snapshot # -- you mount the snapshot
tar cvf /root/mydb-backup.tar /mnt/snapshot
lvremove /dev/vg0/mariadb-backup
systemctl start mariadb # -- starts service again


Restoring a backup
------------------

Logical restore:

mysql -u root -p inventory < /backup/mariadb.dmp
mysql < /home/student/mydb.mysqldump

Physical restore:

systemctl stop mariadb
rm -rf /var/lib/mysql/*
tar xvf /root/mydb-backup.tar -C /