Tuesday, July 31, 2018

Ansible for Windows


How does it work?
----------------

- uses "winrm" module
- control machine can be Linux or Windows subsystem for Linux (WSL) bash shell
- available starting version 1.7

Installing the Control Machine
------------------------------

- needs winrm: pip install "pywinrm>=0.2.2"
- A linux control machine is required to manage windows hosts (can be a WSL bash
  shell)
- Ansible will not move to Windows control machines
- Windows Subsystem for Linux (Beta) is not supported by Microsoft or Ansible
  and should not be used for production systems
- Ansible may also run on Cygwin but might have issues due to Cygwin's
  implementation of fork()

Enabling WSL
1. Follow the instruction on the link below to enable WSL in windows
2. Open bash terminal and execute the ff commands
 
sudo apt-get update
sudo apt-get install python-pip git libffi-dev libssl-dev -y
pip install ansible pywinrm

# this step is only necessary for Windows builds earlier than 16188, and must be repeated each time bash is launched,
# unless bash is launched as ``bash --login``
source ~/.profile

3. You can now start creating the inventories

Authentication Options
----------------------

- can be set via "ansible_winrm_transport" variable

Option        
Local Accounts        
Active Directory Accounts        
Credential Delegation
 

Basic
Yes        
No        
No

Certificate
Yes        
No
No
Similar to SSH where a certificate is assigned to a local user
instead of using a password to authenticate
Kerberos
No        
Yes
Yes
Preferred method than NTLM. You need to install "python-kerberos"
module on control machine (also needs a computer account in Active
directory)
NTLM        
Yes        
Yes
No

CredSSP        
Yes        
Yes
Yes


Kerberos Authentication
-----------------------

Installing python-kerberos dependencies
# Via Yum
yum -y install python-devel krb5-devel krb5-libs krb5-workstation

# Via Apt (Ubuntu)
sudo apt-get install python-dev libkrb5-dev krb5-user

# Via Portage (Gentoo)
emerge -av app-crypt/mit-krb5
emerge -av dev-python/setuptools

# Via pkg (FreeBSD)
sudo pkg install security/krb5

# Via OpenCSW (Solaris)
/opt/csw/bin/pkgutil -U
/opt/csw/bin/pkgutil -y -i libkrb5_3

# Via Pacman (Arch Linux)
pacman -S krb5
Installing python-kerberos
You can install once you installed python-kerberos dependencies

pip install pywinrm[kerberos]

Kerberos is installed and configured by default on OS X and many Linux
distributions.
Configuring Kerberos
1. Specify in krb5.conf the list of primary and secondary domain controllers
 
[realms]

 MY.DOMAIN.COM = {
  kdc = domain-controller1.my.domain.com
  kdc = domain-controller2.my.domain.com
 }

2. Specify the domain you want to access
 
[domain_realm]
    .my.domain.com = MY.DOMAIN.COM
Testing kerberos connection
If krb5-workstation (yum) or krb5-user (apt-get) is installed, you can check if you are
authorised by your domain controller:

kinit user@MY.DOMAIN.COM

Then see if you have acquired a ticket:

klist
Automatic kerberos ticket management
- by default, Ansible 2.3 automatically manages kerberos tickets when both username and
  password are specified for a host that's configured for kerberos
-


Sunday, July 29, 2018

Ansible Configurtion File


Introduction
------------

- ansible will use only 1 config file in the following order
    1. ANSIBLE_CONFIG (an environment variable)
    2. ansible.cfg (in the current directory)
    3. ~/.ansible.cfg
    4. /etc/ansible/ansible.cfg
- prior to version 1.5, order was:
    1. ansible.cfg (in the current directory)
    2. ANSIBLE_CONFIG (an environment variable)
    3. ~/.ansible.cfg
    4. /etc/ansible/ansible.cfg
- comments can be "#" or ";"
    -> `#` cannot be used as inline comment
    -> `;` can be used as inline comment
    -> example
         # some basic default values...
         inventory = /etc/ansible/hosts  ; This points to the file that lists
                                           your hosts
Getting Latest Configuration
----------------------------

- if installed via package manager, it is present inside /etc/ansible
- if via pip, config must be created manually
- example config:

Environment Variables
---------------------

- ansible allows setting of ENV variables
- if present, it will override settings loaded from config
- these are defined in constants.py:

General Defaults
----------------

- settings under [defaults] section

action_plugins = ~/.ansible/plugins/action_plugins/:/usr/share/ansible_plugins/action_plugins
- specifies location of action plugins
- action plugins enables module execution, templating, etc..
- used more by developers
allow_unsafe_lookups
- new in version 2.2.3,:2.3.1
- prevents templating engine in evaluating jinja2 expression (security risk)
- example:
    {{lookup('pipe', '/path/to/some/command', allow_unsafe=True)}}
allow_world_readable_tmpfiles=True
- new in version 2.1
- makes temporary files created on target machines to be world readable
- useful when becoming an unprivileged user
 ansible_managed = Ansible managed

- string that can added on files created from templates
- other modifiers
    * {file} - expands to full path of file
    * {host} - hostname of the target machine
    * {uid} - owner of file created from template
- other example:
    ansible_managed = Ansible managed: {file} modified on %Y-%m-%d %H:%M:%S by {uid} on {host}
ask_pass = True
- determines whether Ansible will prompt for an SSH password
- default: no
ask_sudo_pass = True
- determines whether Ansible will prompt for sudo password
- helpful on systems enforcing sudo passwords
- default: no
ask_vault_pass = True
- determines whether Ansible asks for vault passwords
- default: no
bin_ansible_callbacks = False
- controls whether plugins are loaded when running /ust/bin/ansible
- may be used to log activity from command line, send email notifications, and so on
- always loaded for /usr/bin/ansible-playbook if present and cannot be disabled\
callback_plugins = ~/.ansible/plugins/callback:/usr/share/ansible/plugins/callback
- specifies location of callback plugins
- callback plugins are pieces of codes that get called on specific events to trigger notifications
- mostly used by developers
callback_whitelist = timer,mail
- new in version 2.0
- enables a list of additional callbacks
- cannot change or override default stdout callback   
command_warnings = False
- by default, ansible issues a warning to use the module counterpart of an executed shell command
- e.g using "yum install package" as a shell command instead of using "yum" module
- this setting controls whether warnings are supressed or not

- this can also be appended to the end of command/shell string
    $ ansible all -bm shell -a "git update foo warn=no"
connection_plugins = ~/.ansible/plugins/connection_plugins/:/usr/share/ansible_plugins/connection_plugins
- specifies location of connection plugins
- connection plugins permit to extend channel used by ansible to transport commands and files
- mostly used by developers
deprecation_warnings = True
- new in version 1.3
- allows disabling of deprecating warnings in ansible-playbook output
- deprecation warnings indicate usage of legacy feature that will be removed in future releases
display_args_to_stdout = False
- if enabled, parameters will be displayed together with the task names
    TASK [testing display_args_to_stdout _raw_params=touch /tmp/secret, _uses_shell=True] ***********************
- default is "False:
display_skipped_hosts = True
- if disabled, ansible will not display status for a task that is skipped
- default is True
- note that ansible will always show the task header for any task regardless of whether or not
  the task is skipped
error_on_undefined_vars
- available since version 1.3
- by default, task with undefined variable will be mark as failed and any host for that task
  will be remove from the rest of the play
- if enabled, tasks with undefined variables will not be marked as failed, hosts on that task
  will still be included on the rest if the play, but the variables on the destination will
  be rendered as it is
- example:
 
TASK [testing template] ***************************************************************************************************
changed: [192.168.1.40]

PLAY RECAP ****************************************************************************************************************
192.168.1.40               : ok=1    changed=1    unreachable=0    failed=0  

|master| ~$
|master| ~$
|master| ~$ cat /tmp/testfile
{{ secret_var }}
executable = /bin/bash
- shell to spawn under sudo environment on target system
- default is /bin/sh
- starting version 2.1, this is overwridden by variable "ansible_shell_executable"
filter_plugins = ~/.ansible/plugins/filter_plugins/:/usr/share/ansible_plugins/filter_plugins
- used to extend template system
- mostly used by developers
    
force_color = 1
- sets color mode even running w/o TTY
force_handlers = True
- new in version 1.9.1
- determines if handlers will be ran on host even a failure occure on that host
- default is False
- can also be set per play or on command line
forks = 5
- number of parallel process to spawn when communicating to remote hosts
- default is 5
fact_caching = jsonfile
- if enabled and there is a valiud data for a host, ansible will use that rather than running
  `setup` module
- value should be name of cache plugin
- example values: redis, jsonfile
fact_caching_connection = localhost:6479:0
- tells ansible where to cache facts
- for json, value is path to local directory
- for redis, value is in "localhost:port:database" format
fact_caching_timeout = 86400
- sets expiry of facts
- 0 = disables expiry
- positive values = TTL in seconds
fact_path = /home/centos/ansible_facts.d
- sets custom path for local facts for "setup" tasks
- default is in /etc/ansible/facts.d
    * affects only fact gathering triggered by a play when "gather_facts: True"
gethering = implicit
- new in version 1.6
- controls how control machine gathers facts from remote nodes
- values:
    a. implicit
         - default
         - fact cache will be ignored
         - facts will be gather per play unless "gather_facts: False" is set
    b. explicit
         - inverse of implicit
         - facts will not be gathered unless requested in the play
         - uses fact cache
    c. smart
         - for each new host w/o facts recently discovered will be scanned
         - if a previously scanned host is encountered again, it will no longer be scanned
         - uses fact cache
gather_subset = all
- new in version 2.1
- specifies what set of facts is gathered
    a. all - gather all subsets
    b. network - network info
    c. hardware - hardware info (longest facts to gather)
    d. virtual - info about virtual machines inside the target host
    e. ohai - gather facts from ohai
    f. facter - gather facts from facter
- can be combined in the ways:
    gather_subset = network,virtual  # network and virtual subsets only
    gather_subset = !hardware        # don't gather H/W facts
    gather_subset = !all             # collects minimal amount of facts
hash_behaviour = replace
- new in version 2.0
- controls whether variables encountered in multiple places are merged or replace (default)
hostfile
- deprecated since 1.9
- use inventory instead
host_key_checking = True
- enabled by default in version 1.3 and later
internal_poll_interval = 0.001
- new in version 2.2
- sets interval (in seconds) of Ansible internal processes polling each other
-
lower values improve performance with large playbooks at the expense of extra CPU load
- higher values are more suitable for Ansible usage in automation scenarios, when UI responsiveness
  is not required but CPU usage might be a concern
- default corresponds to the value hardcoded in Ansible ≤ 2.1
inventory = /etc/ansible/hosts
- specifies default inventory file on where to look for hosts
- called "hostfile" in versions before 1.9
inventory_ignore_extensions = ~, .orig, .bak, .ini, .cfg, .retry, .pyc, .pyo
- files to ignores when treating a directory as a source of multiple directories
- eg: ansible -i /home/my_inventories/ webserver -a "uptime"
jinja2_extensions = jinja2.ext.do,jinja2.ext.i18n
- enables jinja2 extensions
- used by developers
library = /usr/share/ansible
- path to search for modules
- can be a colon-separated list of directories
- ansible can also search inside ./library alongside a playbook
- in case of modules with same name, paths are searched in order and the first one encountered is used
- example of having multiple locations:
 
$ mkdir -p /srv/modules
$ cd /srv/modules
$ git checkout https://vendor_modules .
$ git checkout ssh://custom_modules .
$ export ANSIBLE_LIBRARY=/srv/modules/custom_modules:/srv/modules/vendor_modules
$ ansible [...]
local_tmp = ~/.ansible/tmp
- new in version 2.1
- directory to store the file (module + constants + some codes) that will be sent to remote hosts
- default is inside user's home directory
log_path=/var/log/ansible.log
- if present, ansible will log executions on this file
- not enable by default
- even w/o this, ansible logs module arguments to the target system's syslog (excluding passwords)
- user running the playbook must have permissions to write on this file
lookup_plugins = ~/.ansible/plugins/lookup_plugins/:/usr/share/ansible_plugins/lookup_plugins
- allows low-level extensions to be loaded from different locations
- used by developers
merge_multiple_cli_tags
- new in version 2.3
- when True, specifying --tags and --skip-tags more than once will merge all tags altogether
- default in 2.3 is False (only last tag will be used)
- default in 2.4 is True
- after 2.8, the option will be removed (multiple tags will always be merged)
module_lang = en_US.UTF-8
- language used between module and the system
- default is value of $LANG on the controller
- used only if module_set_locale is set to True
module_name = command
- default module (-m) to use for /usr/bin/ansible
- default is "command" module
module_set_locale
- controls whether ansible will prepend locale-specific environment variables
- if enabled, it results in LANG, LC_MESSAGES, and LC_ALL being set when the module is executed on the given remote system
- new in version 2.1 and defaults to True
- in version 2.2, default is False
module_utils = /usr/share/ansile/my_module_utils
- location to look for module utils
- module utils are python modules that is combined to ansible modules when sending to remote machine
- module utils is useful for extracting common code when developing a set of site-specific modules
- multiple locations can be separated by colon
- ansible will also look under ./module_utils alongisde a playbook
nocolor = 0
- colorized the output for success and failure
- default is 0 (ON)
- 1 is OFF
nocows = 0
- by default, ansible will take advantage of cowsay if installed
- can be disabled by using 1 (OFF)
pattern = *
- default group to talk to when "hosts:" stanza is supplied
- default is talk to all hosts
- only applicable to /usr/bin/ansible-playbook
poll_interval = 15
- sets how often to check on status of tasks launch with "async" option
- default is 15 seconds
private_key_file = /path/to/key/file
- path to private key used in SSH connections to remote systems
remote_port = 22
- default port the control machine will communicate to the remote
- can be specified in the inventory as a variable for a host or group of hosts
remote_tmp - ~/.ansible/tmp
- location where to put modules on the remote system
- default is under the remote user's home directory
remote_user = root
- default username ansible will connect as for /usr/bin/ansible-playbook (default is root)
- /usr/bin/ansible will use current user running the ad-hoc command when this setting is not set
restrict_facts_namespace = False
- new in version 2.4
- allows restricting fact on their own namespace rather than pusing them into main
- example:

    When False:
    - debug: var=ansible_system
   
    When True:
    - debug: var=ansible_facts.ansible_system
retry_files_enabled = True
- tells ansible whether a failed playbook should create a .retry file
- default is True
retry_files_save_path = ~/.ansible/retry-files
- path where .retry files are created
- default location is adjacent to the play
- can be changed on any directory which is writable (directory will be created if it doesn't exist)
- "retry_files_enabled" must be set to True to use this setting
roles_path = /path/to/role
- new in version 1.4
- default is to search first under roles/ subdirectory of a playbook project
- can be set to multiple path separated by colon
squash_actions = apk,apt,dnf,homebrew,package,pacman,pkgng,yum,zypper
- new in version 2.0
- allows optimizing actions that call modules that support list parameters when using with_ looping
- instead calling the module once for each item, the module is called once with full list
- default is only for certain package managers
- currently supports only modules with name parameter and only when the item is the only thing being passed to the parameter
stdout_callback = skippy
- new in version 2.0
- overrides the default stdout callback for ansible-playbook
strategy_plugins = ~/.ansible/plugins/strategy_plugins/:/usr/share/ansible_plugins/strategy_plugins
- allows users change the way ansible run tasks on targeted hosts
- mostly used by developers
strategy = free
- specifies the default strategy
sudo_exe = sudo
- sets the path to sudo
sudo_flags=-H -S -n
- additional flags when using sudo support
- default is "-H -S -n" which sets the HOME environment variable
- "-n" will conflict using passwordless sudo auth like pam_ssh_agent
sudo_user = root
- default user to sudo is root
- available only in /usr/bin/ansible
system_warnings = True
- new in version 1.6
- allows disabling of system warnings / potential issues when running ansible
- default is True
- may include warnings about 3rd party packages issues that should be resolved
timeout = 10
- default SSH timeout to use on connection attempts (in seconds)
- which is the default: 10 or 12 seconds?
transport = paramiko
- default transport to use if "-c " is not specified in /usr/bin/ansible or /usr/bin/ansible-playbook
- default is "smart"
    * use openSSH if OS is new enough to support ControlPersist
    * uses paramiko if openSSH w/ ControlPersist is unavailable
- other values are: local, chroot, and jail
vars_plugins = ~/.ansible/plugins/vars_plugins/:/usr/share/ansible_plugins/vars_plugins
- allows low-level extensions to be loaded from different locations
- mosty used by developers
vault_password_file = /path/to/vault_password_file
- new in version 1.7
- sets path as an alternative to "--vault-password-file"
- can also be a script

Privilege Escalation Settings
-----------------------------

- settings under [privilege_escalation] section
- as of version 1.9, "become" supersedes the old "sudo/su"

become = True
- equivalent of adding "sudo:" or "su:" to a play or task
- default is "no" or "False"
become_method = su
- sets privilege escalation method
- default is "sudo"
- other values are: su, pbrun, pfexec, doas, ksu
become_user = root
- equivalent to "ansible_sudo_user" or "ansible_su_user"
- default is root
become_ask_pass = True
- asks for privilege escalation password
- default is False
become_allow_same_user = True
- allows sudoing to same user
- useful for switching SELinux contexts

Paramiko Settings
-----------------

- Paramiko is the default SSH connection on Enterprise Linux 6 and earlier
- settings are located under [paramiko_connection] section

record_host_keys = True
- records newly discovered host fingerprints
proxy_command = ssh -W "%h:%p" bastion
- new in version 2.1
- similar to OpenSSH's "ProxyCommand"
- requires minimum Paramiko version 1.9.0
- in Enterprise Linux 6 this is provided by python-paramiko1.10 in EPEL repo

OpenSSH Settings
----------------

- located under [ssh_connection] section
- OpenSSH is the default connection type for all OS that supports ControlPersist

ssh_args = -o ControlMaster=auto -o ControlPersist=60s
- specifies additional SSH options
- if "-o ControlPath" is set as one of the arguments, "control_path" setting is ignored
control_path=%(directory)s/ansible-ssh-%%h-%%p-%%r
- location to save ControlPath sockets
- for system w/ long names, you may shorten it like: control_path = %(directory)s/%%h-%%r
- on version 1.4 and later, ansible will instruct user to use "-vvvv" in situations where
  long pathnames are encountered
control_path_dir=~/.ansible/cp
- new in version 2.3
- base directory of ControlPath sockets -- %(directory)s
retries = 0
- adds option to retry failed ssh executions (not remote command failure)
- can be used to temporarily relieve network issues
- default is "0"
scp_if_ssh = False
- if set to True, SCP will be used instead of SFTP during ansible connections
- can be used if SFTP is not available on the target system
- dafault is False
pipelining = False
- if set to True, SSH operations required to execute a module on the remote server is reduced
  by executing many ansible modules w/o actual file transfer
- improves performance when enabled
- when using sudo, you must disable "requiretty" in /etc/sudoers on all target hosts
- default is False
ssh_executable="/usr/local/bin/ssh"
- new in version 2.2
- location of SSH binary to use
- this is overriden by "ansible_ssh_executable" inventory variable

Accelerated Mode Settings
-------------------------

- settings under [accelerate] section
- acceleration is a useful performance feature if pipelining cannot be enable in
  your environment
 
accelerate_port = 5099
- port to use for accelerated mode
accelerate_timeout = 30
- new in version 1.4
- controls timeout for receiving data from a client
- if no data is received within this time, socket connection will be closed
- a keepalive packet is sent back to the controller every 15 seconds so this timeout should
  not be set lower than 15 seconds
accelerate_connect_timeout = 1.0
- new in version 1.4
- controls timeout for socket connect call
- should be kept relatively low
- connection to "accelerate_port" will be attempted 3 times before Ansible will fall back to
  ssh or paramiko (depending on your default connection setting) to try and start the
  accelerate daemon remotely
- default is 1.0 seconds
- this valie can be set less than a second but do that only when your LAN speed is fast
accelerate_daemon_timeout = 30
- new in version 1.6
- controls timeout for the accelerated daemon (in minutes)
- default is 30 minutes
- prior to 1.6, this timeout is hardcoded from the time of daemon's launch
- for 1.6+, this timeout is based on the last activity of the daemon and configurable via this
  option
accelerate_multi_key = yes
- new in version 1.6
- if enabled, multiple private keys can be uploaded to the daemon
- any clients connecting to the daemon must also enable this option
- new clients first connect to the target node over SSH to upload the key which is done via a
  local socket file, so they must have same access as the user that launched the daemon
  originally

SELinux Settings
----------------

special_context_filesystems = nfs,vboxsf,fuse,ramfs,myspecialfs
- new in version 1.9
- list of filesystems that require special treatment when dealing with security context
- normal behaviour is for operations to copy the existing context or use the user default
libvirt_lxc_noseclabel = True
- new in version 2.1
- causes libvirt to connect to lcs containers by passing noseclabel to virsh
- necessary when running on systems w/o SELinux
- default behaviour is "no"
show_custom_stats
- new in version 2.3
- if enabled, custom stats (set via set_stats plugin) will be displayed when using default
  callback

Galaxy Settings
---------------

- located under [galaxy] section
 
server
- overrides default galaxy server value of: https://galaxy.ansible.com
- doesn't work against private, hosted repos which Galaxy can use for fetching and installing roles
ignore_certs
- if set to "yes", ansible-galaxy will not validate TLS certificates

Common Tasks
------------

Prevents logging output
to server log file
Add this in ansible.cfg:
no_log = True

Useful when you don't want to log the commands you are executing
on the remote system